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Foreword 


This volume contains the papers presented at the 7th International Workshop on 
Approximation Algorithms for Combinatorial Optimization Problems (APPROX 
2004) and the Sth International Workshop on Randomization and Computa- 
tion (RANDOM 2004), which took place concurrently at Harvard University, 
Cambridge, on August 22—24, 2004. APPROX focuses on algorithmic and com- 
plexity issues surrounding the development of efficient approximate solutions to 
computationally hard problems, and this year’s workshop was the seventh in 
the series after Aalborg (1998), Berkeley (1999), Saarbriicken (2000), Berkeley 
(2001), Rome (2002), and Princeton (2003). RANDOM is concerned with appli- 
cations of randomness to computational and combinatorial problems, and this 
year’s workshop was the eighth in the series following Bologna (1997), Barcelona 
(1998), Berkeley (1999), Geneva (2000), Berkeley (2001), Harvard (2002), and 
Princeton (2003). 

Topics of interest for APPROX and RANDOM are: design and analysis of 
approximation algorithms, inapproximability results, approximation classes, on- 
line problems, small space and data streaming algorithms, sub-linear time algo- 
rithms, embeddings and metric space methods in approximation, math progam- 
ming in approximation algorithms, coloring and partitioning, cuts and connec- 
tivity, geometric problems, network design and routing, packing and covering, 
scheduling, game theory, design and analysis of randomized algorithms, ran- 
domized complexity theory, pseudorandomness and derandomization, random 
combinatorial structures, random walks/Markov chains, expander graphs and 
randomness extractors, probabilistic proof systems, random projections and em- 
beddings, error-correcting codes, average-case analysis, property testing, compu- 
tational learning theory, and other applications of approximation and random- 
ness. 

The volume contains 19+18 contributed papers, selected by the two program 
committees from 54+33 submissions received in response to the call for papers. 
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of the program committees 
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Martin Skutella, Max Planck Institute 

Dan Spielman, MIT 
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Neal Young, UC Riverside 
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Noga Alon, Tel Aviv U. 
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Designing Networks with Existing Traffic 
to Support Fast Restoration 


Mansoor Alicherry'’, Randeep Bhatia’, and Yung-Chun (Justin) Wan? 


' Bell Labs, Lucent Technologies, Murray Hill, NJ 07974 
{mansoor, randeep }@research.bell-labs.com 
> Department of Computer Science, University of Maryland, College Park, MD 20742 


ycwan@cs.umd.edu 


Abstract. We study a problem motivated by a scheme for supporting fast restora- 
tion in MPLS and optical networks. In this local restoration scheme detour paths 
are set-up a priori and network resources are pre-reserved exclusively for car- 
rying rerouted traffic under network failures. (i.e. they do not carry any traffic 
under normal working conditions). The detours are such that failed links can be 
bypassed locally from the first node that is upstream from the failures. This local 
bypass activation from the first detection point for failures along with the dedica- 
tion of network resources for handling failures permits very fast recovery times, 
a critical requirement for these networks. By allowing sharing of the dedicated 
resources among different detours the local restoration scheme results in efficient 
utilization of the pre-reserved network capacity. 

In this paper we are interested in the problem of dedicating the least amount of 
the currently available network capacity for protection, while guaranteeing fast 
restoration to the existing traffic along with any traffic that may be admitted in 
the future. We show that the problem is NP-hard, and give a 2-approximation 
algorithm for the problem. We also show that the integrality gap of a natural re- 
laxation of our problem is (n), thus establishing that any LP-based approach 
using this relaxation cannot yield a better approximation algorithm for our prob- 
lem. 


1 Introduction 


Dynamic provisioning of bandwidth guaranteed paths with fast restoration capability is 
an important network service feature for the emerging Multi-Protocol Label Switched 
(MPLS) networks [7] and optical mesh networks [18]. The fast restoration capabilities 
are required in order to provide the needed reliability for services such as packetized 
voice, critical VPN traffic, etc. Traditionally ring based SONET [11] networks have 
offered 50ms restoration to bandwidth guaranteed services, using pre-reserved spare 
protection capacity and pre-planned protection paths. Pre-planning protection in rings 
has been especially attractive, because of the availability of exactly one backup path 
between any two nodes, leading to very simple and fast automatic protection switching 
mechanisms. However in ring based SONET networks these advantages come at the 
cost of reserving at least half the total capacity for protection. 

A local restoration scheme [12], [16], [20] 1s proposed to provide fast restoration 
in mesh based MPLS and optical networks. In this scheme which is also referred to as 
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link restoration the traffic on each link e of the network is protected by a detour path 
that does not include link e. Upon failure of any link e, any trafficon e is switched to its 
detour path. Thus, link restoration provides a local mechanism to route around a failure. 
In this restoration scheme the restoration capacity of the pre-setup detours is not used 
under normal no-failure conditions (except possibly by low priority preemptible traffic). 
Local restoration when used in conjunction with advanced reservation of the restoration 
capacities and pre-setup detours results in low restoration latency. Pre-provisioned link 
restoration also results in operational simplicity since the detours have to be only provi- 
sioned once for a given network topology and since the online connection routing can 
now be done oblivious to the reliability requirements, using only the resources that are 
not reserved for restoration. 

An important consideration for any fast restoration scheme is to minimize the net- 
work resources dedicated for restoration and hence to maximize the proportion of net- 
work resources available for carrying traffic under normal working conditions. In gen- 
eral, link restoration provides guaranteed protection against only single link failures 
hence the reserved restoration capacity may be shared among the different pre-setup 
detours (since at most one detour may carry restored traffic at any given time). Thus 
pre-provisioned link restoration scheme offers the promise of fast restoration recovery 
for just a small fraction of the total capacity reserved for restoration, due to the high 
degree of restoration capacity sharing that is possible in mesh networks. 

In many situations one would like to support fast restoration on existing networks 
without disturbing the existing traffic, meaning that the restoration scheme can only 
use up to the current available network capacity (link capacity minus existing traffic 
capacity) for protecting the existing traffic and any new traffic. Note that existing traffic 
makes the problem harder. A simpler polynomial-time 2-approximation algorithm in 
the absence of existing traffic was presented in [1] (See Related Work). 

In this paper we are interested in the optimization problem of dedicating the least 
amount of the currently available network capacity for protection, while guaranteeing 
fast restoration to the existing traffic along with any traffic that may be admitted in the 
future, for the pre-provisioned link restoration scheme. Specifically we are interested in 
partitioning the available link capacities into working and protection, such that the latter 
is dedicated for restoration and the former is available to carry any current or new traffic, 
with the objective of guaranteeing link restoration for minimal total protection capacity. 
Note that in a network with a static topology this problem may need be solved only 
once, since the solution remains feasible even as the admitted traffic pattern changes. 
However, the solution may not stay optimal over time as the admitted traffic pattern 
changes, and may be recomputed occasionally to ensure efficient utilization of network 
resources. Also changes in network topology (which are common but not frequent) 
may require recomputing the solution, since the old solution may not even guarantee 
link restoration. 


1.1 Problem Definition 


Given a undirected network G = (V, E), with link capacities ue and existing traffic We 
on link e € E the problem is to partition the capacities on link e into a working capacity 
We and a protection capacity pe (S.t. We + Pe = Ue) Such that 
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— The total protection capacity } ` ep Pe is minimized. 

— For every link e = (u,v), in the network G — e obtained by removing link e from 
G, there exists a path Fe (detour for link e) between nodes u and v, such that every 
link e’ on P, satisfies per > We. 

— The working capacity we is at least the amount of the existing traffic We. 

— In case such a partition is not feasible on G, output an empty solution. 


In other words, on link e, pe capacity is reserved for carrying restored traffic during 
failures and at most we (we > We) traffic is carried during normal working conditions. 
Thus on the failure of link e, at most we traffic on it is rerouted over the pre-setup detour 
path Pe using only the reserved capacities on the links on P. 

Note that given the pe and we values for all links e in any feasible solution, the 
detour paths P, can be easily computed. Hence we do not include the computation of 
the detour paths P, in the statement of the problem. 


1.2 Our Contribution 


We show that given an instance of the problem, it can be determined in polynomial time, 
using a fast and efficient algorithm, if the problem has a feasible solution. However, we 
show that computing an optimal solution for feasible instances is NP-hard. Moreover, 
we present a simple and efficient algorithm that computes a solution to the given feasible 
instance in which the total protection capacity reserved is guaranteed to be within two 
times the protection capacity reserved in any optimal solution. We also show that the 
integrality gap of a natural relaxation of our problemis R(n), thus establishing that any 
LP-based approach using this relaxation cannot yield a better approximation algorithm 
for our problem. 


2 Related Work 


The main approaches for supporting pre-provisioned link restoration scheme in mesh 
networks are based on identifying ring structures. Once the set of rings are identified 
then pre-planned restoration schemes as in SONET [11] are employed. In some of these 
approaches the network is designed in term of rings [17] or by partially using rings [9]. 
Thus, these schemes are only applicable to constrained topologies. In some other of 
these approaches each link is covered by a cycle leading to a cycle cover for the net- 
work [9]. Each of these cycles is then provisioned with enough protection capacity to 
cover the links that belong to it. On the failure of the link the working traffic is rerouted 
over the protection capacities in the surviving links of the covering cycle. There are 
two drawbacks of these approaches: first the amount of pre-provisioned protection ca- 
pacity can be significant and second it is hard to find the smallest cycle cover of a 
given network [19]. An improvement to these schemes is those based on the notion 
of p-cycle [10]. Here the main idea is that a cycle can be used to protect not just the 
links on the cycle but also the chords (spokes) of the cycle, thus showing that far fewer 
cycles (than in a cycle cover) may be sufficient for providing full protection. An algo- 
rithm to minimize the total spare capacity, based on solving an integer program over 
all possible cycles is given in [10]. To the best of our knowledge no fast approxima- 
tion algorithms for this problem are known. An alternative to cycle covers, intended to 
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overcome the difficulty of finding good covers, is to cover every link in a network with 
exactly two cycles [8]. A set of cycles that meets this requirement is called a double cy- 
cle cover [13]. For planar graphs, double cycle covers can be found in polynomial-time. 
For non-planar graphs, it is conjectured that double cycle covers exist, and they are typ- 
ically found quickly in practice. However, even for double cycle cover based protection 
schemes, the required pre-provisioned protection capacity can be significant. 

Non-ring based approaches to link restoration on mesh networks include general- 
ized loop-back [14], [15], where the main idea is to select a digraph, called the primary, 
such that the conjugate digraph, called the secondary, can be used to carry the switched 
traffic for any link failure in the primary. Chekuri et al. [5] consider the problem of 
adding protection capacity to the links of a given network (primary) carrying working 
traffic, at minimum cost, so that the resulting network is capable of supporting link pro- 
tection for a given set of links, where the protection is provided to the working traffic 
on the primary network. In their model no limit is imposed on the total capacities of the 
links, and they provide a 4-approximation algorithm when all links in the original pri- 
mary network have uniform bandwidth (carrying the same amount of working traffic) 
and they provide a 10.87-approximation algorithm for the general case. In addition they 
also provide a O(log n)-approximation algorithm for the problem of jointly designing 
the primary and protected networks, given a demand matrix for the working traffic. 

Our previous work [1] considers the problem under the assumption that there is no 
existing working traffic on any link (We = 0). We also allow the rerouted traffic to be 
split on at most two detours in the event of a failure. We show that the optimization 
problem is NP-hard and provide a 2-approximation algorithm. We also give a lower 
bound for the problem when there is no restriction on splitting of rerouted traffic. 

All the schemes mentioned earlier assume that protection is provided for a single 
link failure. Choi et al. [6] present a heuristic for protecting against two link failures, 
based on link restoration. The problem of survivable network design has also been ex- 
tensively studied [4, 3]. Most of the work here has focused on obtaining strong relax- 
ations to be used in cutting plane methods. 


3 Algorithm 


Recall that ue is the total capacity of link e, and We is the amount of existing traffic 
in link e. Wedefine ue — We to be the maximum protection capacity available on the 
link e. 

Our algorithm (Algorithm A) creates a solution by first computing a maximum 
spanning tree T4 based on the maximum protection capacity available on the links. 
It sets the protection capacity of any link e on the maximum spanning tree TA to the 
maximum protection capacity available on link e. For any non-tree (T4) link e (called 
cross link), it initially sets its working capacity equal to the minimum protection capac- 
ity assigned to the links on the unique path in T'4 between the endpoints of link e. It 
then selects a few of the cross links and boosts up their protection capacities. The main 
idea is to protect the cross links by using only links in the tree T4. Also for each tree 
link e its detour consists of all but one tree link, and the boosting up of some cross links 
is used to ensure that the cross link on this detour has enough protection to protect e. 
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The detailed algorithm is given in Algorithm A. We denote the working and protec- 
tion capacity assigned by Algorithm A on link e by wå and p4 respectively (w4+p4 = 
ue). Conceptually the algorithm (as well as the analysis) consists of two phases — find- 
ing basic protection capacity in all links, and then boosting the protection capacity for 
some cross links. These two steps are combined into one in the algorithm, by setting 
w of cross link e; to max(min(p, ve, — w), We,) in line 18. p is the minimum pro- 
tection capacity of the unique path in the tree, which can protect this cross link. w is 
the maximum working capacity of a link in the unique path, which is still unprotected. 
So we need to assign at least w protection, but cannot have a working capacity of more 
than p in this cross link. Also, since we have to support all the existing traffic, wa has 
to be at least We,. 


Algorithm 1 Algorithm A 


Let {e1,€9,.-.,@m} be the links sorted in decreasing order of maximum available protection 
capacities (ue — We). 
Ta =o 
fori= 1,... m { 
if (Ta U {e;} does not form a cycle) { 
Ta = TAU {ex} 
if (Wa; > we, /2) 
No solution exists. 
we. = Wy 
De, =W wé 
Mark e; as unprotected 
} else { 
Let P = {e;,,€35,...,€;, } be the unique path in T4 connecting the endpoints of 
ej. 
Let M be the links in P which are marked as unprotected. 
w= maxeem wA (w= 0ifM = 0) 
p = MiNeeP Pe 
if (p < We;) 
No solution exists. 
wet = max(min(p, ue; — w), We; ) 


No solution exists. 
Unmark edges in M. 


} 

} 

if any link e; is marked 
No solution exists. 


3.1 Correctness 


Lemma 1. The solution (if any) returned by Algorithm A is a feasible solution. 


Proof. It is easy to see that the amount of working traffic of every link (w4) in any 
solution returned by Algorithm A, is always at least the amount of existing traffic (We). 
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Now we show that for every link e = (u, v), there is a backup path P, in G — e between 
nodes u and v, such that every link e’ on P satisfies ps > wê. 

Case 1, for alle € T4: Note that G—e is connected, because otherwise link e would stay 
marked at the end of the algorithm in which case no solution is returned by Algorithm 
A. Thus, there is at least one cross link ee = (te, Ue), Which, together with the unique 
path in T4 between nodes ue and ve (excluding link e), forms a backup path for e. 
Without loss of generality, let €e be the first cross link considered by the algorithm, 
such that adding it to T4 results in a cycle C containing link e. Consider the path 
Pe = C \ {e}. The link ee has been assigned the least protection by Algorithm A 
among the links on Pe. Note that link e is marked (hence in M) at the time when ee is 
considered by the algorithm. Thus when link e, is considered by the algorithm we must 
have pA > w > W. = w2 implying that P, is a valid backup (detour) for link e. 

Case 2, for all e ¢ T4: The backup path P, for link e is the unique path in T4 connect- 
ing e, which always exists. The links on this path have enough protection because the 
algorithm sets the workingtraffic w/ of link e to at most mingep, på. 


Lemma 2. Ifa feasible solution exists, the algorithm will return a solution. 


Proof. The algorithm will not return a solution in 4 cases. 

Case I, there exists a link e € T4, We > ue/2: If a tree link e has We > ue/2, then 
no solution exists. This is because, the maximum protection capacity available on e 
(ue — We) is strictly less than ue/2. If there was a solution, then there must exist a path 
P, in G — e between the end points of e all whose links e’ have maximum protection 
capacity available ue — We > u./2. But then 7’4 is not a maximum spanning tree 
based on the maximum protection capacity available on the links, a contradiction. 
Case 2, there exists a link eg € T4, p < We: The proof for this case uses arguments 
similar to case 1. 

Case 3, there exists a link eg € T4, w > p. : Note that in this case M Æ Ø. Let 
ec = (uc, vc) and let Pe, be the unique path in T4 connecting nodes us and ve. Let 
e € M be a marked link on the path P., with w = We = we, at the time when €e is 
considered by the algorithm. Let P, be a feasible detour for link e. Thus the maximum 
protection capacity available on all links e’ on Pe is ue — We > w. Also Pe U {e} 
forms a cycle in G, thus at least one link on FP, is a cross link for T4, that forms a cycle 
containing link e when added to T4. This link must have been considered before link 
€e since it has strictly more maximum protection capacity available on it. Hence link e 
must already be marked before link e, 1s considered, a contradiction. 

Case 4, there is an unmarked link e = (u,v) at the end: In this case in G — e nodes u 
and v are not connected. Hence, no detour is possible for link e. 


From the above two lemmas, we have the following theorem. 
Theorem 1. Algorithm A is correct. 


Remark: For each link e in T4, we can lower its protection capacity until lowering it 
further would require decreasing the working capacity of some other links, or would 
make the working capacity of link e so large that there is no feasible detour for it. This 
trimming may reduce the total protection capacity however, it has no implication on our 
analysis of the worst-case approximation ratio for the algorithm. 
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4 Analysis 


In this section we show that A is a 2-approximation algorithm. First we define some 
notations. Let p??* be the protection capacity of link e in an optimal solution OPT, 
and w2P* be the working capacity of link e in OPT, thus w2?? = ue — p2P?. To 
compare the solution of Algorithm A with OPT, we construct a maximum spanning 
tree Topr based on the protection capacities per T 

We partition the links in the network into 4 sets: Eao = {ele € T4,e € Topr}, 
Exo = {ele ¢ Ta,e ¢ Tort}, Eao = {ele € Ta, e ¢ Topr}, and Exo = {ele ¢ 
T4,e € Topr}. We define per T to be > jen p2?* (sum of protection capacities 
in OPT over all links in E), på to be È CE på (sum of protection capacities in our 
solution over all links in E), and ug to be $e m7 ue. For a link e ¢ Ty, we use BZ to 
represent the unique path in T4 connecting the endpoints ofe. Similarly fore ¢ Topr, 
we use BOPT to represent the unique path in Topr connecting the endpoints of e. 
Since both T4 and Topr are spanning trees, both trees have the same number of edges. 
It is easy to see that |E46| = |E qo}. 

For a cross link e (not in T4}, we assume that the Algorithm A assigns a basic 
“Level-1” protection capacity of přt = max(0,ue — minge ga på) and a working 
capacity of wf! = min(ue, minere pa på). The cross links e with pt! = p4 are non- 
boosted cross links. Note that the protection capacity of the remaining cross links e 
are boosted from pt to We: by Algorithm A, where e’ is the tree link with the largest 
existing traffic (We), among all tree links protected by e (which are in M when link e is 
considered by the algorithm). Also note that there are at most n — 2 such boosted cross 
links, each protecting a different link in T4. It is easy to see that p < ee A p2 + 
D egTa pe T cers We. 

The total protection capacity used by Algorithm A is thus at most p4 iaiT på ber 


L1 L1 
PEz0 + PE 56 + Weio + We.o: 
, : `- „OPT < pOPT A 
Lemma 3. For all links e45 € E40, We, 5 S Pe» for any e € Borr- 


Proof. For all links e45 € E46, (note that e 46 is a cross link for Topr) 


Woe < min pOPT where S is the backup (detour) path for e 45 in OPT (1) 
e€ 
< min pe” (2) 
ec BOPT 
AO 
OPT OPT 
< po * ,foranye’ € Bo- . (3) 


The first inequality follows from the definition of S. The second inequality holds be- 
cause Topr is a maximum spanning tree of OPT based on the protection capacities 


OPT 
Pe 
PT 


Lemma 4. For all links e ïo € Exo, Pee < MiNneepa på. 
“AO 


Proof. Note that e zo is in the OPT tree Topr, but not included in T4. Therefore the 
maximum protection possible on e go in any solution is at most the minimum protection 
assigned by Algorithm A to the links on the unique path in T4 connecting the endpoints 
of link e zo. 
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Lemma 5. For any links eag E Ego, and any links ejo E€ Eo where ezo lies 
on the unique path in Topr connecting the endpoints of link e4, we have eee < 


—— 


OPT 
Uezo = Wez ` 
Proof. 
Li _ . A 
Pero = max(0, uezo — MineeBA, Pe ) (4) 
OPT 
OPT 


The first equality follows from the definition of Pero: The second inequality holds by 
Lemma 4 and the fact that uezo 2 Dae T The third inequality holds by Lemma 3, with 
e' = ejo. 

Lemma 6. For any links e46 € Eg, and any links €z9 € Ego where € Zo lies on 
the unique path in To pr connecting the endpoints of link e 46, we have De å +We,o + 


mo T. 
Proof. 
PT 
Pe Wega tréig S tejo (tego = Uso ) (7) 
= port 4 uezo (8) 
£02, F (9) 


The first inequality follows from Lemma 5. The second equality follows from the defi- 
nitions of protection and working traffic. The third inequality follows from the fact that 
p2PT must be at least ue/2 for any link e € Topr. The proof is along the same lines 


as the proof of Case 1 in Lemma 2. 


L1 PT $ WPL 


A O 
Lemma 7. pē o + We. t PEro S PEae Exo" 


Proof. For each link e,g € E46, we will show how to pair it up with a unique link 
ezo E€ Ejo, where e J lies on the unique path in Topr connecting the endpoints of 
link e 45. Summing Lemma 6 over all such pairs, the lemma follows. 

Consider a bipartite graph (E a5 UE 49, Em), where Em contains edge (€ 46, €40) 
if e zo lies on the unique path in Jopr connecting the endpoints of link e4g. A one 
to one pairing of edges in E46 with edges in E ĝo can be done by finding a perfect 
matching on the bipartite graph. We now show a perfect matching must exists, using 
Hall’s Theorem [21]. Given any subset S$ of £46, find a forest F which is the union 
of all the unique paths in Topr connecting the endpoints of links in & 4,6. Note that 
all links in F are in Topr. It is easy to see that at least |S} links in F are not in TA. 
Otherwise we can create a cycle in T4 involving a link in S and links of F which are 
in T4. Therefore links in S may be paired up with at least |S} links in £40, and the 
condition in Hall’s Theorem is satisfied. 


Theorem 2. A is a polynomial time 2-approximation algorithm, i.e., p% tes PE, aT 


1 OPT 
Pix + PE x6 + Wesco + Wags S 2PE > 
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a From Lemma 7 we have PB ao + We, + pẹ < pei? T4 2pRET. T. Also 
Pho t Wesco = UEso < 2pG*7 since OPT has to oe at least 5 the capacity of 


each link in E4o for protection (see Lemma 6). Also p% pee | Bao r’ because there is 


Ll 


no solution in which the working capacity of a cross link e ġġ can exceed tego ~ Pezo: 


The proof follows. 


4.1 Worst Case Example 


We give a worse case example to show that the analysis is almost tight. Consider a ring 
of size n with unit capacity, the optimal solution is to put 0.5 units of protection on each 
link. This gives a total protection of % units. Our algorithm would return a tree of size 
n — 1 and put full protection to each link, which gives a total protection of n — 1 units. 
Therefore, the approximation factor is close to 2. 


5 LP-Based Approach 


In this section we study a natural relaxation to our problem which can be modeled as a 
linear program. In this relaxation on the failure of a link its traffic can be arbitrarily split 
and rerouted over multiple detours. That is, in this relaxation we require that the graph 
G — e must support at least we flow between nodes u and v, for every link e = (u,v), 
using only the protection capacities on the remaining links. The LP is shown below. 

For the LP, we transform the undirected graph into a directed graph by replacing 
each link into two directed links. We use the notation pair(e) to denote the link in the 
opposite direction of link e. We assume that both the directed links fail together. Note 
that in this relaxation on the failure of link e, at most we traffic on it is rerouted over 
multiple detour paths P, using only the protection capacities on the links on the paths 
P.. Weuse flow(e,i) to denote the amount of flow rerouted on link 2 on the failure of 
link e. The relaxed LP is 


Minimize >> sc p(Pe/2) 
Subject to the following constraints 


5 flow(e, i) = pS flow(e,i) Vec kE, veV (10) 


t€ bin (v) t€ bout (V) 
flow(e, pair(e)) = we Vec E (11) 
flow(e,e) =0 Vee E (12) 
flow(e,i) < pi Ve,i € E,e Æ pair(i) (13) 
we> We  VecE (14) 
De + We = Ue Vec E (15) 
We = Wpair(e) Vec E (16) 


Note that since the protection capacity of each (undirected) edge is counted twice, once 
for each direction, we divide the total protection capacity by 2 in the objective function. 
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Constraint 10 takes care of the flow conservation at each node (including the end nodes 
ofe). For link e = (u,v), a flow of we from node u to node v is guaranteed by sending 
a flow of we from v to u (constraint 11) and using the flow conservation equations at 
nodes wand v. Constraint 12 guarantees that this flow conservation is not achieved using 
the link (u, v). Constraint 13 guarantees that the flow does not exceed the protection 
capacity of any link. Constraint 14 guarantees that the working capacity of a link is at 
least the existing traffic. Constraint 15 guarantees that sum of protection and working 
capacity does not exceed the total capacity. Constraint 16 is for guaranteeing a solution 
for the undirected graph. 

The results of [1] imply that this LP has an integrality gap of 2 when the existing 
working traffic We on all links is zero. We show however that when We is not restricted 
then this gap is R(n). The proof of the following lemma appears in the full paper [2]. 


Lemma 8. The integrality gap of the LP or the ratio of the optimal integral solution to 
the optimal relaxed solution is R(n). 


6 NP-Hardness 


In this section we show using a reduction from 3-SAT that the decision version of the 
problem to pre-provision existing networks to support fast restoration with minimum 
over-build (PREPROVISION) is NP-hard. 

Let {X1, Xo,...,Xn} be the set of variables and {C,, Co,..., Cm} be the set of 
clauses in a given instance C of 3-SAT. We construct an instance of PREPROVISION 
such that it has a feasible solution of total protection capacity at most 4.5n + 24m if 
and only if C is satisfiable. As shown in Figure 1, for each variable X; we create a 
ring consisting of 6 unit-capacity links (arranged like a hexagon) without any existing 
traffic. We have a unit-capacity link splitting the ring into two equal halves. This link, 
with existing traffic of 1 unit, is represented by a thick segment in Figure |. Three of 
the links, on the left part of the ring, correspond to literal X;, and the other three links, 
on the right part of the ring, correspond to literal X,. For each clause C; we create 
an octopus-like structure. In the center we have a unit-capacity link łc, with existing 
traffic of 1 unit (represented by a thick segment in Figure 1). Three pairs of feet, where 
each pair corresponds to a literal in the clause, are attached to the two endpoints of link 
lc,. Each foot consists of 6 serially-connected unit-capacity links without any existing 
traffic (represented by a dash segment). If literal X; appears in clause C}, one foot 
connects the upper endpoint of link lc, to the node between the first and second link 
of the half-ring corresponding to X;. The other foot of this pair of feet connects the 
lower endpoint of link fc, to the node between the second and third link of the half-ring 
corresponding to X;. Therefore, we have 6n + 32m nodes and 7n + 37m links in the 
network. 

Note that if each foot in the octopus consists of only two links (rather than 6 links), 
we can still prove that the 3-SAT instance is satisfiable if and only if there exists a feasi- 
ble solution to the PREPROVISION instance with total protection capacity of 4.5n+8m 
(with 6n + 8m nodes and 7n + 13m links). However the analysis is much more com- 
plicated and thus we present the proof with 6 links. 
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Fig. 1. Gadgets 


Lemma 9. Ifthe 3-SAT instance is satisfiable, then there exists a feasible solution to 
the PREPROVISION instance with total protection capacity of 4.5n + 24m. 


Proof. If a literal X;(.X;) is true in the satisfying assignment, we put protection of 1 
unit on each of the three links in the half-ring corresponding to X;(X;). Otherwise we 
put protection of half units on each of these three links. Therefore a total of 4.5 units of 
protection is reserved for each variable. In each clause, one of its literals must be true. 
We put protection of 1 unit on each of the 12 links between the clause and one of the 
true literals. We put protection of half units on each link in the remaining two pairs of 
feet (each pair of feet consists of 12 links). Therefore a total of 24 units of protection is 
reserved for each clause. For links with an existing traffic of 1 unit (the thick segments), 
we have no choice but to put 0 units of protection. The total protection capacity reserved 
is therefore 4.5n + 24m. 

It is easy to see that with this reservation all links are protected by a detour path. The 
link which splits a ring in a variable is protected by the 3-link half-ring corresponding 
to the true literal for the variable. Since there is at least one true literal in each clause, 
the link lc, in the middle of the gadget of a clause is protected too. The working traffic 
in the remaining links is either zero or half, and therefore they can be protected because 
all the remaining links have protection capacity of half units. 


Due to lack of space, we give the proof of the other direction in the full paper [2]. 


Theorem 3. The problem of pre-provisioning existing networks with minimum over- 
build is NP-hard. 
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Abstract. We consider the problem of Simultaneous Source Location — 
selecting locations for sources in a capacitated graph such that a given 
set of demands can be satisfied. We give an exact algorithm for trees and 
show how this can be combined with a result of Racke to give a solution 
that exceeds edge capacities by at most O(log? n log log n), where n is 
the number of nodes. On graphs of bounded treewidth, we show the 
problem is still NP-Hard, but we are able to give a PTAS with at most 
O(1+e) violation of the capacities, or a (k+ 1)-approximation with exact 
capacities, where k is the treewidth and € can be made arbitrarily small. 





1 Introduction 


Suppose we are given a capacitated network and we have various demands for 
service within this network. We would like to select locations for servers in order 
to satisfy this demand. Such problems arise naturally in a variety of scenarios. 
One example of this would be the placement of web caches in the Internet, or 
file servers in an intranet. Another example would be choosing the locations of 
warehouses in a distribution network. 

What does it mean for a server to “serve? a demand? Most previous work 
has assumed that each server can service each demand for some cost (typically 
this cost is linear in some underlying distance between server and demand) [20, 
16, 3]. In some cases the servers have been considered to be capacitated (each 
one can provide for only some number of demands) [17]. Still, the primary goal 
can be considered as minimizing the aggregate distance of demands to servers. 

In many natural applications, there is no meaningful notion of distance. Con- 
sider serving a web page across the internet. The latency (travel time of a single 
small packet) under low-congestion conditions tends not to be noticeable to the 
end-users. The real difficulty here is the underlying capacity of the network. If 
links become congested, then latency will increase and throughput will suffer. In 
the case of a distribution network, there may be some relation (typically non- 
linear) of costs to distance traveled. But we will definitely have to consider the 


* The authors want to thank the Aladdin Center. This work was in part supported by 
the NSF under grants CCR-0085982 and CCR-0122581. 


K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 13-26, 2004. 
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available throughput! The transportation network has capacities as well as costs, 
and previous work (assuming costs only) tended to ignore this constraint except 
at the warehouses themselves. 

We consider the problem of Simultaneous Source Location (SSL) — selecting 
locations for sources in a capacitated network in order to satisfy given demands. 
Our goal is to minimize the number of sources used. Arata et al. previously gave 
an exact algorithm for the Source Location problem, in the scenario where the 
sources must be able to satisfy any single demand [2]. They also show that the 
Source Location problem is NP-hard with arbitrary vertex costs. In our problem 
we must satisfy all demands simultaneously (thus the name Simultaneous Source 
Location). This is a better model of the natural problems described above. Si- 
multaneous Source Location is easier than the Source Location problem with 
arbitrary vertex costs explored by Arata et al. — our version of the problem can 
be reduced to theirs using a high-cost super-sink that is satisfied if and only if 
all other demands are met. However, we show that our version of the problem is 
NP-Hard as well and describe various approximations for it. 

Our results take several forms. We describe techniques for solving Simultane- 
ous Source Location on a variety of simple graphs. We give an exact algorithm 
on trees and an approximation for graphs of bounded treewidth. We observe 
that, in contrast to many other NP-Hard problems (for example vertex cover), 
Simultaneous Source Location is still NP-Hard even on graphs of treewidth two. 
Using our algorithm for trees combined with a result of Racke et al. [18,6] and 
later Harrelson et al. [11], we show how to solve source location on a general 
undirected graph while overflowing the capacity by an O(log” n log log n) factor. 
Combining this with a hardness result for directed graphs allows us to show 
that no tree decomposition similar to Räcke’s decomposition can be found in 
the directed case. We show Simultaneous Source Location is at least hard on 
general undirected graphs, but there remains a significant gap between the best 
approximation algorithm and the known lower bound. 


2 Problem Statement 


An instance of Simultaneous Source Location (SSL) consists of a graph G = 
(V, E) along with a capacity function u : E — Rt on the edges and a de- 
mand function d : V — Rt on the vertices. We must select some subset of the 
vertices S C V to act as sources. A source set is considered to be feasible if 
there exists a flow originating from the nodes S that simultaneously supplies 
demand d, to each node v € V without violating the capacity constraints. This 
is single-commodity flow — we can imagine adding a single “super-source”’ that is 
connected to each node of S with an infinite capacity edge, and a single “super- 
sink” that is connected to each node of V with an edge of capacity d,, and asking 
whether the maximum flow equals the sum of the demands. Our goal is to select 
such a source set S of the smallest size. 

From the above description, it is clear that the problem is in NP. A source set 
S can be checked for feasibility by simply solving a flow problem. Unfortunately, 
finding the set § of minimum size is NP-Hard. 
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At times we consider various generalizations of this problem. For example, 
we might have certain nodes in V that are not permitted to act as sources, or 
have costs associated with making various nodes sources and seek to find a set S$ 
of minimum cost. We will also consider simplifications of the problem that place 
restrictions on the graph G (for example by bounding the treewidth). 


3 Solving Simultaneous Source Location on Trees 


Suppose our graph G = (V, E) is a tree. This special case allows us to solve SSL 
exactly by dynamic programming. For each vertex v and number of sources 2, 
we define f (v,i) to be the amount of flow that must be sent to v by its parent in 
order to satisfy all demands in the subtree of v, assuming this subtree contains 2 
sources. Our algorithm assumes that the tree is rooted and binary; in general we 
can create an equivalent binary tree by creating virtual nodes and connecting 
them by edges of infinite capacity, as shown in Figure 2. For convenience, for 
each node v we define u(v) to be the capacity of the edge from v to its parent. 
Our algorithm is described in Figure 1. 


Algorithm Binary Tree(G, d) 


l. Initialize f(v,z) = co for all v € V and 0< i< |V] 
2. For each leaf vertex v € V: 
(a) Set f(v, 0) = dv. 
(b) Set f(v, i) = —u(v) for all i > 1. 
3. Consider any vertex v with children vı, v3 for whom f has been computed: 
(a) Loop over all values of 7; and iz with 0 < 7,72 < |V]. 
(b) If f(v, i1) < u(vi) and f(v2,i2) < u(v2) then: 
i. Set f(v, i1 +i2) = min( J (v, i; +i2), mar( f(v, i1)+ f (vz, i2)+ do, —u(v))). 
ii. Also set f(v, i1 + iz + 1) = —u(v). 
4. Continue until f is defined at all vertices. 
5. Return the minimum k such that f(r,k) < 0 where r is the root. 


Fig. 1. Algorithm for SSL on a Binary Tree 


Assuming that the above algorithm computes f(v, i) correctly for each vertex, 
the correctness of the result is immediate. We need to show that f(v, i) correctly 
represents the minimum amount of flow that must be sent from the parent of v 
provided the number of sources in the subtree is 2. 


Theorem 1. The algorithm described produces an exact solution to the source 
location problem on a binary tree. 


Proof. The proof will be by induction. Our base case is at the leaves. Either a 
leaf is a source or it is not. If the leaf v is a source, then it requires no flow from 
its parent, and can send at most u(v) flow upwards along its edge. This yields 
f(v,1) = —u(v). On the other hand, if the leaf is not a source it requires flow dy 
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(its demand) from the parent, so f(v,0) = dy. Of course, it might not be feasible 
to send this amount of demand if d, > u(v). 

We now consider any node v. Suppose we have correctly computed f(v1, 71) 
and f(v2,t2) for all values 21,72 for the children of node v. Suppose we would 
like to compute f(v,z). There are 2 sources to be placed in this subtree. If v is 
not a source itself, then all the sources are in the child trees. The total demand 
sent into v will have to be enough to satisfy the demand d, and additionally to 
satisfy any residual demand on the children. This means f(v,7) = min(max(d, + 
f (v1, ii) + f(v2,i— i1), —u(v))) where the minimum is over choices of 7; and the 
“max” term ensures that a child node cannot provide more flow to its parent 
than the capacity of their connecting edge. This is exactly what will be computed 
in the algorithm, in step 3b(2). Notice that if satisfying either child tree in this 
way would require overflowing a capacity u(v1) or u(v2) then this allocation 
of sources to subtrees is not feasible and so should not be considered; this is 
resolved in step 3b. However, it is also possible that v is itself a source. In this 
case, provided there is some choice of 77 <7—1 such that f(vı, i1) < u(vi) and 
f(v2,i— i1 — 1) < u(ve), we will be able to produce a solution. This solution can 
send u(v) upwards since v itself is a source. This is dealt with in step 3b(ii). It 
follows that the algorithm computes exactly the correct values f(v,i) and the 
algorithm solves SSL. 


Theorem 2. We can solve SSL in time O(n?) on a tree, even if some nodes 
are disallowed as sources. 


If our tree is non-binary we can replace any node with more than two children 
by multiple nodes as shown in Figure 2. This increases the number of nodes by 
at most a constant factor. 





Fig. 2. By adding additional nodes linked by infinite capacity, we can convert any 
non-binary tree into an equivalent binary tree by only doubling the number of nodes 
in the tree 


We can also modify our algorithm to disallow certain nodes as sources. If a 
leaf v cannot be a source, then we ignore step 2b and instead set f(v,7) = dy for 
all ¢. If some higher node cannot be a source, then we remove step 3b(i1) for that 
node (this step considers the case where v is a source). The correctness proof is 
the same as before. 
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4 Using Racke’s Result 


In recent work, Harold Racke showed that for any undirected graph, it is pos- 
sible to construct a tree which approximately captures the flow properties of 
the original graph[18]. Later work [6,11] improved the result and described a 
polynomial-time algorithm to construct such a tree. The most recent result is 
stated more precisely in the following theorem: 


Theorem 3. Given any capacitated, undirected graph G = (V, E,u : E — R*), 
there exists a capacitated tree T = (Vr, Er,u : Ep — Rt) with the following 
properties: 


I. The vertices of V are the leaves of T. 

2. For any multicommodity flow F which is feasible on G, there exists a flow 
of equal value between the corresponding leaves on T. 

3. For any flow Fr feasible between the leaves of T, there is a feasible arr flow 


on G for some p = O(log? nlog log n). 


This gives us an approximation for the source location problem. We first 
construct a tree T as described above. We then solve SSL on the tree, permitting 
only the leaves to act as sources, using the algorithm of Section 3. We consider 
using the sources we have obtained on the original graph. We know there exists 
a flow Fr on the tree from our selected source nodes which satisfies all the 
demands. It follows that ey is feasible on the graph. We conclude that if we 


violate the capacities by a factor of p = O(log? n log logn), then we have a 
feasible solution to source location. On the other hand, any feasible solution on 
the graph must also be feasible on the tree. This allows us to produce an exact 
solution (in terms of the number of sources) while increasing the capacities by 
O(log’ n log log n). 


Theorem 4. We can produce an optimum (in terms of number of sources) so- 
lution to SSL in polynomial time, if we permit O(log? nloglogn) stretch on the 
capacities. 


Our results also have interesting implications for directed graphs. Consider 
the possibility of a Racke-like representation of a directed graph by a tree, where 
the capacity on an edge when routing “upwards” might differ from the “down- 
wards” capacity. Suppose such a thing existed, and could be computed in poly- 
nomial time, for some factor p. This would enable us to solve SSL exactly on 
directed graphs, while exceeding capacities by a factor of p. But this is NP-Hard, 
as will be shown in Section 6.2. Thus we have the following: 


Theorem 5. No Rdcke-like decomposition of a directed graph into an p-appro- 
rimately flow-conserving tree can be computed in polynomial time, for any value 
of p polynomial in n. 


Note that our hardness result is a computational hardness result and assumes 
a tree-like structure decomposition. Azar et al. [5] have found an existential 
hardness result which is true for any decomposition, but their bound is weaker: 


O(n). 
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5 Stmultaneous Source Location with Bounded Treewidth 


5.1 Defining Treewidth 


The notion of treewidth was introduced by Robertson and Seymour [19]. Many 
problems that are in general intractable become polynomial-time solvable when 
restricted to graphs of bounded treewidth. Furthermore, many graphs arising 
from natural applications have bounded treewidth. A good survey on the topic 
is given by Bodlaender [7]. Here is one of the many equivalent definitions of 
treewidth: 


Definition 1. A graph G = (V, E) has treewidth k if there exists a tree T = 
(V-, E+) along with a mapping f : V- = 2” with the following properties: 


1. For alla € Vz, \f(a@)| <k +1. 

2. For any (u,v) € E there exists some a € V, such that u,v E f(a). 

3. For any a, ß,y € Vr where B lies along the path from a to y, if for some 
x EV wehave z € f(a) and z € f(y), then we must also have x € f(). 


The above conditions essentially state that each tree vertex represents some 
subset of at most & graph vertices, each edge in the graph has its endpoints 
represented together in at least one of the tree vertices, and the set of tree 
vertices which represent a single graph vertex must form a contiguous subtree. 

We observe that it is possible to produce such a tree decomposition for a 
graph of treewidth k in time linear in the number of nodes and vertices (but 
exponential in k). Assuming k is constant, we are able to produce a tree de- 
composition — and thereby implicitly detect graphs of constant treewidth — in 
polynomial time. In general, however, computing the treewidth of an arbitrary 
graph is NP-Hard. 


5.2 Nice Decompositions 


Bodlaender [8] also introduced the notion of a nice decomposition and proved 
that any tree decomposition of a graph can be transformed into a nice decom- 
position still of polynomial size. In a nice decomposition, each node a@ € V, has 
one of the following types: 


— A leaf node & has no children, and |f(a@)| = 1 

— An add node a has one child 8 with f(a) = f(8) U{v} for some node v € V 

— A subtract node a has one child 6 with f(a) = f(8) — {v} for some node 
v € f(B) 

— A merge node a has two children 8, y with f(a) = fF(8) = f(y) 


In addition, the nice decomposition has a root node p (which is a subtract node) 
with f(p) empty. 
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5.3 Approximation for Graphs of Bounded Treewidth 


Suppose we are given a graph G = (V, E) with treewidth k, for which we would 
like to approximate the SSL problem. Our algorithm takes in some set of current 
sources S along with a graph (V, E) and returns a set S’ of sources which are 
feasible for the given graph. Our algorithm for this problem appears in Figure 3. 


Algorithm SL(S,V,E) 


1. Check whether the source set S is feasible for (V, E); if so return S 
2. If not, find sets X and By that have the following properties: 
(a) |Bx|<k+1 
(b) For all (x,y) € E with z € X and y E€ V — X, we have z € Bx 
(c) S(](V — X) is not a feasible source set 
(d) SU(V — X) U Bx is a feasible source set 
3. Recursively solve SL(S |J Bx, (V — X) U Bx, E) 


Fig. 3. Algorithm for SSL with treewidth k 


We claim that the set SË of returned sources has |SË| < (k + 1)|.S*| where 
|S*} is the smallest feasible set of sources for (V, E) which includes the given set 
S as a subset. 


Lemma 1. Assuming we are always able to find sets X and Bx with the prop- 
erties described, algorithm SL is a (k +1)-approzimation for the source location 
problem. 


The prove is done by induction on the number of sources required by the opti- 
mum solution. 

Of course, we still need to prove that we can always find the sets X, Bx with 
the required properties in polynomial time. In a general graph, such a pair of 
sets might not even exist, but we will use the assumption of treewidth k to prove 
existence and the ability to find the sets in polynomial time. 


Lemma 2. If the current set of sources S is not feasible for the graph G = 
(V, E) with treewidth k, then there exists a pair of sets X, Bx with the required 
properties; furthermore, such a pair of sets can be found in polynomial time. 


Proof. We produce a tree decomposition (7, f) of G. For each tree node a, we 
define Ta to be the subtree of r rooted at a. Wedefine f(T) = Uger, f(A). For 
each node œ we will test whether S|()(V — f (Ta)) is a feasible set of sources. We 
find node a such that S\J(V — f (Ta)) is infeasible, but such that S|(J(V — f(7,)) 
is feasible for each child 8 of a. Note that such an œ must exist; we simply 
travel upwards from the each leaf of the tree until we find one. We now consider 
returning X = f(t.) and Bx = f(a). We will show that these sets satisfy the 
required properties. 
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Since the graph has treewidth k we know|Bx| = |f(a@)| < k+1. Consider any 
(x,y) E€ E with z € X and y € V — X. From the definition of treewidth, there 
must exist some node 8 with x,y € f(8). Since y is not in f(Ta) we conclude 
that is not in Ta. On the other hand, there is some node y € 7, such that 
x € f(y) (this follows from z € X). The path from y to 8 must pass through a, 
so the treewidth definition implies z € f(a) and therefore x € Bx as desired. 
The selection of a guarantees that S\J(V — X) is not a feasible source set. This 
leaves only the fourth condition for us to prove. 

We consider the children of a. A pair of them y1, Y2 must have f(y) N f (72) 
€ f(a) because of the contiguity property of tree decomposition. Thus the sets of 
nodes represented by the subtrees of the children can intersect only in the nodes 
of Bx = f(a). We know that for each child y, the set of nodes SUJ(V — f(1,)) 
would be feasible. It follows that we can cover all the demand of f(7,) using 
nodes of S and nodes external to the set. Since the children sets are disjoint 
except for nodes which we have declared to be sources, the flows to satisfy each 
child subtree are edge-disjoint; any flow from external nodes must also pass 
through Bx, and we conclude that we can cover f(T) using S| Bx. It follows 
that we can cover all of X using the sources S|] Bx, making SU Bx U(V — X) 
a feasible source set for the entire graph. 


Theorem 6. Algorithm SL produces ak+1-approzimation to the SSL problem 
on a graph of treewidth k. 


5.4 Bounded Treewidth with Capacity Stretch 


We will describe an exact algorithm for the SSL problem on graphs of bounded 
treewidth. The running time of this algorithm will be exponential in the tree- 
width, and also depends polynomially on the maximum edge capacity. In the 
general case where the capacities may be large, we will use the technique of 
Appendix A to obtain a solution with 1 + e stretch on the edge capacities. 

Suppose we have found a nice tree decomposition (7, f). We will construct a 
set of vectors of dimension & + 3. Each vector has the following form: 


(a, t, fi; fa, ee) fii) 


Here a € V;,0 < i < |V|, and the f; are feasible flow values (we assume 
these are from a polynomially-bounded range of integers). Let Sg = f (Ta)— f(a) 
represent the nodes represented by the subtree rooted at œ minus the nodes of 
its boundary (the nodes of f(a) itself). A feasible vector represents the excess 
flow needed to be sent directly from the nodes of f(a) to Sa to satisfy all the 
demand in Sx if Sẹ contains 7 sources. 

We observe that if k is a constant and flow is polynomially-bounded, then 
the number of possible vectors is polynomial in size. We will determine which 
such vectors are feasible in polynomial time, then use this to solve SSL. Our 
algorithm is as follows: 
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1. Start with the empty set of feasible vectors. 
2. Consider each node @ from the bottom up: 

— If a is a leaf, then add vector (a, 0,0). 

— If ais an add node with child 8, then for each feasible vector for 8, copy 
that vector for a, placing flow 0 in the new position corresponding to 
the additional node in f(a) — f(). 

— If a is a merge node with children 6,7 then for each pair of feasible 
vectors £g, £y for the children, create a vector for œ: ta = rg + Ly 
(adding the number of sources and the flows while changing the choice 
of node from V, to a). 

— If @ is a subtract node with child 8, then consider each feasible vector 
xg for the child. Let the subtracted node be b € f(@) — f(a). This node 
requires some flow rẹ which is the sum of the demand dẹ and the flow 
value for b in xg. We consider all feasible allocations of flow F(a) to 
nodes a € f(a) such that |F'(a)| < u(b,a) and dae ray F(a) 2 re. For 
each such allocation we construct a vector £ẹ whose number of sources 
is equal to xg and with flow value at a equal to the flow value in xg plus 
F(a). This corresponds to refusing to make b a source. We now consider 
making b a source. This corresponds to creating a vector £a with one 
more source than the vector xg. We set the flow value for a node a € f(a) 
to be the flow value in zg minus u(b, a). 

3. Now consider all vectors for the root node p. These are simply pairs (p, 7) 
since f(p) is empty. We return the minimum value of ¿ such that (p,2z) is in 
the feasible set. 


Theorem 7. If there are F possible flow values, the above BTW algorithm runs 
in time O(kn? N F2*+2) where n = |V| and N = |V;| and k is the treewidth. 


This running time is polynomial assuming that F is polynomial and k is a 
constant. Ifthe number of possible flow values is not polynomial, we can use the 
result of Appendix A to effectively reduce the number of flow values. This will 
cause us to exceed the graph capacities by a factor of 1 + e. 


5.5 Lower Bound for Treewidth 2 Graphs 
We show that the SSL problem is NP-Hard even on graphs with treewidth two. 
Theorem 8. SSL is NP-Hard even on graphs with treewidth two. 


Proof. The proof is by reduction from subset sum. We are given a set of numbers 
{£1, £2;,..., Zn}, and would like to determine whether some subset of the given 
inputs sums to A. Suppose the sum of all the numbers is S. We construct an SSL 
instance with 2n +2 nodes. Each number will be represented by a pair of nodes 
of demand S$ with an edge of capacity S between them. Our example is a four 
level graph. On the first level we have a node of demand A which connects to one 
side of every pair of nodes that represent a number. The capacity on the edge 
between the A node and the number x; node is exactly z;. The number nodes 
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Fig. 4. Simultaneous Source Location is NP-Hard even on graphs of treewidth two. 
To satisfy each demand in this graph using only n sources we must find a partition of 
{£1, £2,- .-, £n} whose sums are A and S -— A 


from the second level are paired with the nodes from level 3. All nodes from level 
3 are connected to a single node at level 4 with an edge of capacity corresponding 
to their number. The node at level 4 has demand S — A. This graph is shown 
in Figure 4. If there exists a subset of the numbers summing to A, then we 
can place sources on the lefthand node for each of the numbers in that subset 
and the righthand node for all the other numbers; it is straightforward to see 
that this is a feasible SSL making use of n sources. On the other hand, consider 
a SSL solution. We must select one of the two nodes for each of the numbers 
(otherwise there is not enough capacity to satisfy them). It follows that the SSL 
uses at least n sources. If exactly n sources are used, then the result corresponds 
to a subset sum solution. It follows that solving source location exactly on this 
graph will solve subset sum. The graph given has treewidth two; we can see this 
because if we remove the node of demand A, the remaining graph is a tree. We 
take the (treewidth one) tree decomposition and add the node of demand A to 
the subset f(a) for all a. This is a tree decomposition of width two. 


6 Simultaneous Source Location on Directed Graphs 


6.1 Greedy O(logn) Approximation 


We are given a directed graph G = (V, E) for which we would like to approx- 
imate the SSL problem. We propose a simple greedy algorithm. We start with 
no sources and no demand satisfied. We add the source which maximizes the in- 
crease in the total satisfied demand. We repeat this until all demand is satisfied. 


Theorem 9. The greedy algorithm gives an O(log n) approximation on the num- 
ber of sources with no violation of the capacities. 


Proof. Suppose that the optimum solution uses t sources. At some point in time, 
suppose our current set of sources can cover demand d and the total demand 
in the graph is D. Consider the residual graph after adding the flows from our 
sources to satisfy demand d. The residual demand is D — d, and if we were to 


Simultaneous Source Location 23 


add all the optimum sources we would be able to satisfy the full residual demand 
(note that this is essentially single commodity flow since sources are equivalent). 
It follows that we can add one source to the residual graph to satisfy demand 
bod We apply the standard greedy analysis for problems like SETCOVER to 
show that the full demand will be covered in O(log D) steps. Assuming that the 
maximum and minimum demand are within a polynomial factor of one another, 
we are done. Otherwise, we can apply scaling arguments of Appendix A to give 
the desired O(log n) factor. 


6.2 Lower Bound for Directed Graphs 


We show that O(log) is the best approximation factor we can expect to obtain 
for the directed SSL problem in polynomial time, due to a reduction from set 
cover. 


Theorem 10. We cannot approximate directed SSL to better than O(log n) in 
the worst case, unless NP C DTIM E(n? 8 8 7)), 


Proof. Suppose we would like to solve a set cover instance. We construct a graph 
with one node for each set, one node for each element, and one additional node. 
The additional node is connected by a directed edge of capacity one to each of 
the sets. Each set is connected by a directed edge of capacity N, where N is 
more than the sum of the number of sets and elements, to each of its elements. 
The additional node has demand one, each set has demand one, each element 
has demand N. We solve the SSL problem to some approximation factor p on 
this graph. We first observe that no element should be selected as a source; 
otherwise we could simply select one of the sets containing that element as a 
source instead. Second, we observe that the additional node will be selected as 
a source. We have a solution consisting of pt nodes, where t is the optimum. 
Consider the set nodes that we selected as sources. Every element node must be 
connected to one of these set nodes in order for its demand to be satisfied (note 
N is greater than the number of sets). It follows that we have a set cover of 
size pt — 1. Similarly, observe that any set cover, plus the additional node, forms 
a feasible SSL solution. So we have obtained a pt—1 > p approximation to set 
cover. This is unlikely for p smaller than log n due to the results of Feige [10]. 


We observe that even if we are allowed to violate edge capacities by some 
large factor (say less than N), the reduction from set cover still holds. 


7 Lower Bound for Undirected Graphs 


We show that SSL does not have a polynomial-time approximation scheme via 
an approximation-preserving reduction from vertex cover. 

Assume we can approximate SSL on any graph to a constant œ. Now for an 
instance of Vertex Cover on a graph G=(V,E), we will setup an instance of SSL. 
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For every vertex the demand is equal to its degree and all edge capacities are 
unit. 

The main observation is that a vertex cover set and a source location set 
on this graph are equivalent. It is easy to see that a feasible vertex cover is a 
feasible source location: for every edge at least one end is in the vertex cover 
and therefore a source. Thus every vertex will have its demand satisfied. On the 
other hand a source location set can’t have a two hop path over a non source 
vertex, because this vertex will be starved. Hence every edge has at least one end 
in the source location set, 1.e. the source location set is a feasible vertex cover. 

Using the assumption we can approximate the SSL problem on G. Therefore 
we find a set SCV which covers all edges in E and is within a of the optimal 
Vertex Cover. However as Hastad [12] showed Vertex Cover is inapproximate if 
a < 7/6 (although an approximation factor better than 2 would be a surprising 
improvement on existing vertex cover results). 


Theorem 11. Simultaneous Source Location is 1.36067 —e hard on general undi- 
rected graphs if edge capacities are not violated. 


The proof follows from the reduction above and the recent hardness results 
by Dinur and Safra [9]. 


8 Conclusions 


We define the Simultaneous Source Location problem and solve the problem 
exactly on trees. We present a (1+ e€) violation of the capacities PTAS for graphs 
of bounded treewidth. On general graphs we find a solution with exact number of 
sources which can exceed the capacities by at most a factor of O(log? n log log n). 
We show a O(logn) factor approximation on the number of sources with no 
violation of the capacities for general directed graphs. We believe that many 
interesting applications of this problem involve graphs of low treewidth; many 
of the connectivity graphs of real networks have been observed to have low tree 
width [8]. 

The main open problem is the approximability of SSL on undirected graphs of 
large treewidth. No constant approximation on the number of sources is known, 
even if we allow constant violation of the capacities. The only lower bound on 
approximability with exact capacities is 1.36067 — e. An approximation factor 
asymptotically better than 2 would be a surprising improvement on existing 
vertex cover results [15]. One can also consider adding costs on the edges and/or 
the vertices. 
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A Dealing with Super-Polynomial Capacities 


Some of our analyses assume that there are only a polynomial number of capac- 
ities. Here we present a method to extend those analyses to when the capacities 


26 Konstantin Andreev et al. 


are super-polynomial, allowing us to satisfy demands while exceeding capacities 
by a 1 + factor in those cases. 

When capacities are super-polynomial, we express the flow and capacity of 
each edge as aF"* for some value of i (which might be different from edge to edge) 
and some value of œ which is between F and F?. This can be done by rounding 
every flow up to the nearest value. Once this is done, we might no longer have a 
feasible flow. The new capacities (which might be larger than the old capacities 
by al + + factor) will not be exceeded. However, flow conservation may no 
longer hold. Consider any node. If the original inflow was f then the outflow was 
f also. But it is possible that after this rounding upwards, the inflow is still f 
and the outflow has increased to f (1+ Ł). We consider such a flow to be feasible. 
This means that a node might effectively “generate” some amount of flow, but 
this amount is at most 1 of the inflow. 

Now consider any graph with a feasible flow under the representation de- 
scribed above. We would like to transform this into a real flow. If we consider 
splitting the “generated” flow equally among outgoing edges, we will see that 
as flow passes through a node the percentage of “real” flow might decrease by a 


factor of For In the worst case, some edge might have a fraction of “real” flow 


equal to (y >1— F. 

We let F = 2. It follows that we can satisfy at least 1 — € of each demand 
while exceeding capacities by a factor of 1 + =. Since we can scale the flows and 
capacities, this means we can satisfy the demands while exceeding capacities by 
1 + factor. 

This is useful in bounding the running time and approximation factor of 
various techniques for approximating SSL. 
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Abstract. In many economic settings, convex figures on the plane are for sale. 
For example, one might want to sell advertising space on a newspaper page. Self- 
ish agents must be motivated to report their true values for the figures as well as 
to report the true figures. Moreover, an approximation algorithm should be used 
for guaranteeing a reasonable solution for the underlying NP-complete problem. 
We present truthful mechanisms that guarantee a certain fraction of the social 
welfare, as a function of a measure on the geometric diversity of the shapes. We 
give the first approximation algorithm for packing arbitrary weighted compact 
convex figures. We use this algorithm, and variants of existing algorithms, to cre- 
ate polynomial-time truthful mechanisms that approximate the social welfare. We 
show that each mechanism achieves the best approximation over all the mecha- 
nisms of its kind. We also study different models of information and a discrete 
model, where players bid for sets of predefined building blocks. 


1 Introduction 


The intersection between Micro-Economic theory and Computer-Science theory raises 
many new questions. These questions were studied recently by researchers from both 
disciplines (see, e.g., the surveys in [13,6]). A leading example for a problem in this 
intersection is the Combinatorial Auction problem. In a combinatorial auction, a finite 
set of heterogenous items is for sale, and each selfish agent has a valuation for every 
subset of these items. As the auction designers, we try to find an allocation of the items 
among the agents that maximizes the “social welfare” (1.e., a set of disjoint packages 
that maximizes the sum of valuations) or at least to find a good approximation. 

In this paper, we study a variant of combinatorial auctions: e.g., a newspaper wants 
to sell an advertising space on a newspaper page. Agents might have different prefer- 
ences about their desired space: the size of the ad, its location on the page, whether its 
figure is rectangular, square, or elliptic etc. Each agent submits a bid for her favorite 
figure, and we try to find the allocation that maximizes the social welfare. The underly- 
ing packing problem is known to be NP-hard, even for the very simple case of packing 
2x2 squares ([15]). Thus, we settle for a computationally-efficient mechanisms that 
approximate the social welfare. 


* The authors are grateful to Noam Nisan for many helpful discussions, and to Ron Lavi for his 
comments on an earlier draft. 
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In our model, the plane (R?) is for sale. Let N be a finite set of agents (|N| = n). 
Each agent has a private non-negative valuation v; € R* for a single compact convex 
figure si and for any figure that contains it (other figures have a valuation of 0). Every 
agent submits a bid for her desired figure (e.g., an advertiser might want the lower half 
of the first page in a newspaper). After receiving all bids, the auctioneer determines a 
set of winning agents with disjoint figures and the payment that each agent should pay. 
Note that the agents demand figures in fixed locations in the plane, and the auctioneer 
cannot translate or rotate them. Bidding for convex figures is common in many real- 
life scenarios that involve “geometric” bids, e.g., selling real-estate lots, newspaper ads 
and spectrum licenses in different locations. In most existing real-estate or advertising 
auctions, agents are forced to bid on predefined disjoint figures. This might result in 
inefficient allocations that can be avoided by allowing the agents to bid for arbitrary 
figures (which in turn makes the computational problem harder). 

Note that the problem addressed in this paper is more than just finding an algorithm 
with a good approximation ratio. The agents in our model are selfish, and they may 
report untruthful information if this is beneficial for them. We want to design incentive- 
compatible mechanisms, in which each agent uses a strategy that is best for her own 
selfish interest (a dominant strategy), and yet, a certain approximation for the social 
welfare is guaranteed. A general scheme for achieving a welfare-maximizing incentive- 
compatible mechanism is the family of Vickrey-Clarke-Groves (VCG) mechanisms (see 
[11] for a review). However, for implementing such mechanisms we must allocate the 
goods optimally (otherwise it will not be truthful [14]). Thus, since finding the optimal 
allocation is an NP-hard problem, we must find incentive-compatible mechanisms that 
are non-VCG. Almost all the non-VCG mechanisms currently known are for models 
where the agents hold a single secret value (single-parameter models)’. 

In light of these impossibilities, we assume that each agent is interested in a single 
figure. Lehmann et al. [10] initiated the study of the Single-Minded Bidders model for 
combinatorial auctions. Our model is unique since the bids have some common geomet- 
ric properties (e.g., convexity), and also because we actually auction an infinite (even 
uncountable) number of goods (the points in the plane). 

We differentiate between two models of information (similar to the differentiation 
done in [12,3]). In the first model, the auctioneer knows which figure each agent wants, 
but does not know how much this agent is willing to pay for this figure. This model is 
the Known Single-Minded (KSM) model, and this is indeed a “single-parameter” model. 
In the second model, called the Unknown Single-Minded (USM) model, both the fig- 
ures and the values are unknown. In the KSM model we should motivate the agents to 
truthfully declare their true values, where in the USM model the agents might submit 
untruthful bids both for their desired figures and their values’. 


l Actually we prove our results for a more general model that allows non-convex bids as well, 
with some more general restrictions. 

2 Recent results ([9]) show that in many reasonable settings, essentially no IC mechanisms exists 
for multi-parameter models, except the family of weighted VCG mechanisms. 

3 Note that the USM model does not fit into the “single-parameter” definition, since the agents 
have both their values and their figures as their secret data. 
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Another differentiation we make is between a continuous and a discrete model. In 
the continuous model, each agent is interested in an arbitrary compact convex figure in 
R*. For example, if a piece of land is for sale, each agent can draw an arbitrary figure on 
the map. In the discrete model, the plane contains predefined atomic building blocks (or 
tiles), and each agent is restricted to bids for a set of building blocks which are contained 
in some convex figure. For example, if we wanted to resell the island of Manhattan, the 
basic building blocks would be street blocks bounded between two consecutive streets 
and two consecutive avenues. These blocks are typically convex, though not necessarily 
rectangular (e.g., because of the diagonal Broadway Avenue). 


Related Work: 

Our research relates to a sub-field of Micro-Economics called Mechanism Design, 
which studies ways to design mechanisms that encourage agents to behave in a way 
that results in some desired global properties (see, e.g., [11]). Nisan and Ronen [14] 
introduced this concept to CS by the name Algorithmic Mechanism Design. 

Weighted packing of rectangles in the plane was studied in several papers. 
Hochbaum and Maass [7] proposed a shifting strategy for a special case of square pack- 
ing, and generalizations for arbitrary squares appear in, e.g., [5,4]. Khanna et al. [8] 
used similar methods in a model where axis-parallel rectangles lie in a n x n grid. 
They presented an algorithm that runs in polynomial time and achieves an O(log(n))- 
approximation for the optimal welfare. However, it is an open question whether a better 
approximation (and in particular, a constant approximation) exists for this problem. 


Our Contribution: 
We measure the quality of the approximations achieved by mechanisms in our model 
according to an aspect ratio R, which measures how diverse are the dimensions of 
the figures demanded by the agents. R is defined as the ratio between the maximal 
diameter of a figure and the minimal width of a figure (formally defined in Section 2). 
For different families of figures, we construct IC mechanisms, either for the KSM or for 
the USM models. This mechanisms are also individually-rational, i.e., agents will not 
pay more than they value the figure they receive (if any). Therefore, our approximation 
improves as the dimensions of the figures become closer’. 

We study three different families of figures: compact convex figures, rectangles, and 


axis-parallel rectangles. For convex figures in the USM model, we achieve an O(R 3 )- 
approximation to the social welfare. If the bids are restricted to rectangles (not neces- 
sarily axis-parallel), we achieve a better approximation of O(R). 

If the agents bid for axis-parallel rectangles, we can use a slight modification of the 
algorithm due to Khanna et al. [8] to design an IC mechanism that achieves an approx- 
imation ratio of O(log(R)) (the best known approximation ratio for this problem). 

We also present a novel allocation algorithm that achieves an O(R)-approximation 
for packing arbitrary compact convex figures, and as far as we know this 1s the first 
approximation algorithm for this problem. We use this algorithm for constructing an IC 
mechanism, with the same approximation ratio, for the KSM model. 


* For instance, when all figures are disks with the same radius up to a constant, our mechanisms 
achieve a constant approximation. 
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The incentive-compatible mechanisms we present for the USM model are based on 
a family of greedy algorithms presented by Lehmann et al. [10]. For standard combina- 
torial auctions, Lehmann et al. normalized the values by |.S|*, where |S] is the number of 
items in the bundle S and @ is some real constant, and then run a simple greedy alloca- 
tion algorithm on the normalized values. They showed that choosing @ = 4 guarantees 
the best polynomial approximation ratio (unless NP = ZPP). 

We present mechanisms, called &-greedy mechanisms, that normalize the values 
using the geometric area of the figures. That is, for œ € R we assign a normalized 
value of 75 to a bid with a value v for a figure with a geometric area q. We show 


e. . . . l 
that, somewhat surprisingly, for compact convex figures the optimal value for a is 3, 


resulting an O(R3 )-approximation. The difference between the results of Lehmann et 
al. and ours derives from the different divisibility properties of packages in the two 
models. In their model, a finite set of goods is traded, and for a package to intersect 
“many” disjoint packages, its size must be “large”. However, in our continuous model, 
a small package can intersect many disjoint packages. 


l l l 4 

For our discrete model, we present a mechanism that achieves an O(R3) approx- 
imation. However, if the ratio between the minimal width of a figure and the sizes 
of the building blocks (we denote by Q) is smaller than the aspect ratio R, we can 


achieve a better approximation of O(R - Q*'), by running the o-greedy algorithm with 
É log(R 5 

eS 2log(R)+log(Q) — a l l l l 

The paper’s organization: Section 2 describes our model. Section 3 describes our 


results for the USM model, both for the continuous case and the discrete case. Section 
4 presents the results for the KSM model, and Section 5 concludes with a discussion of 
future work. All proofs are given in the full version of the paper ([2]). 


2 Model 


Let B denote the family (set) of bids (figure-value pairs) of the agents®, that is B = 
{(si,vi)|i € N}. Let F denote the family of agents figures, that is F = {s;|i € N}. Given 
a family of bids B, we aim to maximize the social welfare, i.e., find a collection of non- 
conflicting bids (bids for disjoint figures) that maximizes the sum of valuations. For a 
subset C C N of agents with disjoint figures, denote the value of C by V(C) = Yiec vj. 
We denote the set of disjoint figures that achieves the maximal welfare by OPT (to 
simplify the notation we assume that there are no ties, so there is a single optimal 
solution), i.e., 


V(OPT) = max V(C) 
CEN] Vi,jEC siNsj=0 


> Thus, if one can embed the goods of a traditional combinatorial auction as building-blocks in 
the plane, such that each agent bids for building-blocks contained in some convex figure, then 
our approximation scheme improves the approximation ratio achieved in [10]. 

© Since all the mechanisms we consider are truthful, we use the same notation for the secret 
information and the declared information (bid), except of the IC proofs. 
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Definition 1. A mechanism consists of a pair of functions (G,P) where: 


— G is an allocation scheme (rule) that assigns a figure in T (where T is the set 
of compact convex figures in the plane) to every agent such that the figures are 
disjoint, i.e. G(B) € T” and for every i £ j in N, G;(B) A G;(B) = 0 (where we 
denote the figure received by agent i by G;(B)). 

— P is a payment scheme, i.e for any B, P(B) € R”. Denote the payment paid by agent 
i by P;(B). 





All allocation rules we present in the paper, allocate to an agent either her requested 
figure or the empty figure. All the payment rules we consider are normalized, that 1s, 
a losing agent pays zero. Additionally, each agent pays a non-negative payment. We 
assume quasi-linear utilities and that the agents have no externalities (the utility for 
each agent does not depend on the packages received by the other agents), 1.e., the 
utility of each agent i is u;(B) = v;(G;(B)) — P;(B). The agents are rational, so each 
agent chooses a bid that maximizes her own utility. 

A mechanism is incentive-compatible (IC) if declaring their true secret information 
is a dominant strategy for all the agents. In the KSM model, it means that for any 
set of values reported by the other agents, each agent cannot achieve a higher utility 
by reporting an untruthful value, i.e., Vi VB_i Vv; ui((si, vi), B-i) > ui((si, v5), B_i), 
where B_; denote the family of all bids except i’s bid. In the USM model, IC means that 
each agent’s best strategy is to report both her figure and her value truthfully, regardless 
of the other agents’ reports, i.e., Vi VB; Yvi, s; ui((si, vi), B_i) > ui((si,v;),B_i). 

An incentive-compatible mechanism is also individually rational (IR) if for any 
agent i, bidding truthfully ensures him a non-negative utility. That is, Vi VB_; 
u;((si, vi), B_i) > 0. 


Geometric Definitions: 

We state our approximation bounds as functions of few geometric properties of the 
family of figures the agents bid for. We use standard definitions of diameter and width 
of compact figures in KR: 

The diameter d; of a compact set z is the maximal distance between any two points 
in the set, i.e. dz = MaXp,,p.€z||P1 — P2\|2 (||p1 — p2||2 is the Euclidean distance between 
pı and p2). The width w; of a compact set z is the minimal distance between the closest 
pair of parallel lines such that the convex set z lies between them. 








Definition 2. Given a family of figures F in RÊ, the maximal diameter L is the maximal 
diameter of a figure in F, and the minimal width W is the minimal width of a figure in 
F. The aspect ratio R is the ratio between the maximal diameter and the minimal width. 
That is, L = maxzer dz , W = miner wz , R= &. 


The aspect ratio describes how diverse is the family of figures with respect to the fig- 
ures’ diameter and width’. The approximations our mechanisms achieve are asymptotic 
functions of the aspect ratio R. 


1 For example, if all the figures are disks with the same radius, then R = 1. If we have disks of 


diameter 10 and 5x2 rectangles, then R = 9 ==, 
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Denote the geometric area of a compact figure z by q(z). We assume that the di- 
ameter, width and area of any agent’s figure are polynomial-time computable. We also 
assume that given any two agent’s figures, we can decide if the two figures are disjoint 
in polynomial time’. 


The Discrete Model: 
In the discrete model, there is a set of atomic building blocks (we call tiles) embedded 
in the plane. Each agent desires a bundle of tiles that are exactly the ones that are fully 
contained in some compact convex figure, and she reports this figure and her value 
for the set of tiles contained in it. We assume that all tiles have similar dimensions 
(specifically, each tile contains a disk of some positive diameter Wo and its diameter is 
at most 2Wo). Two agents are non conflicting if there is no tile which is fully contained 
in the two figures they report. 

For a given family of bids, we define the width-ratio Q = a The width-ratio gives 
an upper bound on the width of any figure, with respect to the size of the tiles. Clearly, 
we can assume that Q > 1. 


The Greedy Mechanism: 
Lehmann et al. [10] presented the following family of greedy mechanisms for combi- 
natorial auctions: 

Given a family of bids and some function f on the figures, such that f assigns a 
positive real value to any non empty figure (w.Lo.g. all figures are non empty), the 
greedy allocation algorithm picks an allocation ALG, by the following scheme: 

Create a list of the bids sortedfrom high to low according to their values normalized 
by f (i.e, Fis) > 76 >... > HAL 

While the list is not empty, choose a figure s; for which the normalized value is 
highest in the remaining list (with a consistent tie breaking). Add i to the allocation 
ALG and update the list by removing all bids for figures that intersect Sj. 

The specific algorithm is determined by the choice of the function (or norm) f. 
Lehmann et al. suggested using the norm pa for combinatorial auctions, where |s| 
is the size of the package s and & is some non-negative constant. We generalize this 
method for compact figures in R* and define the ot-greedy algorithm to use the norm 
ae where q(z) is the area of figure z’. 


Definition 3. The a-greedy mechanism is a mechanism which uses the -greedy algo- 
rithm as its allocation scheme, where a winning agent ipays according to the following 
payment scheme: 


Let j be the first agent to win, among all the agents whose figures intersects agent 


Aye 
i’s figure, when the greedy algorithm runs without i. If such j exists, i pays aed 


otherwise i pays 0. Losing agents pay 0. 
The properties of the &-greedy mechanisms, proved in [10], also hold in our model: 


8 Note that for polygons the above assumptions hold, and that any compact convex figure can 
be approximated (as good as one wants) by a polygon. 

? For example, the 0-greedy algorithm sorts the figures according to their values and the 1- 
greedy algorithm sorts the figures according to their value per unit of area. 
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Theorem 4. (essentially due to [10]) For every Q, the a-greedy mechanism is polyno- 
mial time, individually rational and incentive compatible for agents bidding for com- 
pact figures in the USM model. 


3 The Unknown Single-Minded Model 


This section considers the problem of designing a polynomial-time, individually- 

rational and incentive-compatible mechanisms, which guarantee some fraction of the 
social efficiency for the USM model. We study three families of figures: convex figures 
and rectangles in the continuous model, and convex figures in the discrete model. We 
use the &-greedy mechanism to create mechanisms with the desired properties for the 
three families. For each family, we find the value of œ that optimizes the asymptotic 
approximation for the social welfare, over all &-greedy mechanisms. For convex figures 


; . 4 eee 

in the continuous model, we show that & = } achieves an O(R3 )-approximation for the 
social welfare. For rectangles in the continuous model, we improve the above result and 
show that & = 5 achieves an O(R)-approximation for the social welfare. Finally, for 


convex figures in the discrete model, we show that a careful choise of & as a function 


of R and Q, gives an approximation ratio between O(R3) and O(R). The proofs of all 
above results are based on a single general result presented in the full version [2]. 


3.1 Convex Figures and Rectangles in the Continuous Model 


The following lemma presents a lower bound on the approximation ratio that can be 
achieved by &-greedy mechanisms, by presenting two constructions that are in a sense 
the hardest inputs for these mechanisms. 


Lemma 5. The O-greedy mechanism for compact convex figures achieves an 
Q(R2U-®)) approximation for anya <1,and an Q(R!+“)-approximation for anya > 

4 
0. Therefore, the a-greedy mechanism for compact convex figures achieves an Q(R3 )- 
approximation. 


Proof sketch: Each of the lower bounds is achieved by a construction that can be built 
for any R large enough. The left part of Figure 1 illustrates the construction used to 
prove the Q(R2"-) bound and the right part is used to prove the Q(R'*%) bound. In 
the left example, a large rectangle contains @(R7) small disjoint squares with a side 
of length W. On the right example, a small disk intersects O(R) disjoint triangles, with 
equal area of @(WL). In both constructions, there is one figure z (filled by small vertical 
lines) that is chosen by the greedy mechanism, while the socially optimal mechanism 
chooses a family of disjoint figures (small horizontal lines), each intersects z. The value 
of z is chosen such that its normalized value FO is a bit greater than 1, and the rest of 
the figures have a normalized value of 1. The value for œ that minimizes the worst case 


approximation is therefore 7 yielding an Q(R3 ) lower bound. 0 
Next, we show that the 4-greedy mechanism achieves an O(R3 )-approximation 


(which, by the above lemma, is the best over all the a-greedy mechanisms). To prove 
this result, we use few elementary geometric properties of convex figures. First, for any 
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Fig. 1. Approximation bounds for the a-greedy mechanism 


compact convex figure z, g(z) = O(d,w;). Additionally, the perimiter of z(denoted by 
Pz) is contiguous, and pz; = ©(d,) (the constants in both cases are independent of the 
figure z). These properties are sufficient for the approximation to hold. 


Theorem 6. When agents bid for compact convex figures in the plane with an aspect 
ratio R, the 4 - greedy mechanism achieves an O(R3)-approximation, and this is the 
best asymptotic approximation achievable by an a&-greedy mechanism (for any a). This 
mechanism is individually rational and incentive compatible for the USM model, and it 
runs in polynomial time. 


Proof sketch: By Theorem 4 the mechanism is IR, IC and it runs in polynomial time. 


Next, we present the idea behind the proof of the O(R3 )-approximation ratio. By the 
definition of the &-greedy algorithm, any figure is either a winner or intersects a winner. 
It is the hardest to prove the approximation bound if the set of winners in the optimal 
solution OPT is disjoint to the set of winners ALG picked by the o-greedy algorithm. 
We map each agent x € OPT to a winner z € ALG that intersects x. We then bound the 
sum of values of any disjoint set of agents’ figures that intersects z, by partitioning them 
to figures that are contained in z and to figures that are not. We use Holder inequality 
and some simple geometric properties of convex figures to show that the upper bounds 
fora = i, for both the contained figures and the rest of the intersecting figures, match 
the lower bounds presented in Lemma 5. 0 


If agents are only interested in rectangles (not necessarily axis parallel), than we 
can derive a stronger result of an O(R)-approximation for the social welfare. While 
the construction of a large rectangle containing Q(R”) small rectangles (as presented in 
Lemma 5) is still possible, the second construction is not. For rectangles, it is impossible 
for a small rectangle to hit (intersect but not contain) many disjoint rectangles. 


Theorem 7. When agents bid for rectangles in the plane with an aspect ratio R, the 5 
- greedy mechanism achieves an O(R)-approximationfor the social welfare, and this is 
the best asymptotic approximation achievable by an a-greedy mechanism (for anya)”. 
This mechanism is individually rational and incentive compatible for the USM model, 
and it runs in polynomial time. 


10 We actually prove a stronger statement. We show that Q(R)-approximation is the best over all 
the greedy mechanisms that sort the bids according to some function of the value and the area 
only (specifically, this includes the function Wr): 
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3.2 Convex Figures in the Discrete Model 


We now turn to look at the discrete model. We first define the mechanism we use for 
the discrete model, we then present the mechanism properties". 


Definition 8. The Discrete Model Greedy Mechanism is a mechanism that given bids 
for compact convex figures in R? in the discrete model, does the following: If Q > R then 


it runs the 4-greedy mechanism, and ifQ < R then it runs the a* - greedy mechanism 
x log(R) 
fora’ = 2log(R)+log(Q)° 


Theorem 9. Consider that the agents bid for compact convex figures in RŻ in the dis- 
crete model, with an aspect-ratio R and a width-ratio Q. Then, the Discrete Model 
Greedy Mechanism achieves an O(R 3 )-approximation for the social welfare. More- 
over, when Q < R it achieves a better approximation of O(R - Q% ). This mechanism 
achieves the best asymptotic approximation among all the mechanisms that choose & 
as a function of R and Q, and in particular it is asymptotically better than the &-greedy 
mechanism for any &. Additionally, the mechanism is IR and IC for the USM model, 
and it runs in polynomial time. 


4 The Known Single-Minded Model 


In this section, we present polynomial-time mechanisms for different families of figures 
in the Known Single-Minded (KSM) model (where the auctioneer knows the desired 
figure of each agent, but does not know her value for the figure). We start by presenting 
an auction for general compact convex figures. We achieve an O(R)-approximation for 


the social welfare, which is better than the O(R3 \-approximation that we proved for the 
USM model. Next, we present a mechanism (based on an algorithm from [8]), which 
gives an O(/og(R))-approximation for axis-parallel rectangles”. 


4.1 Mechanisms for Convex Figures 


Consider the mechanism called the “Classes-by-Area 1-Greedy Mechanism” (CBA-1G 
mechanism) presented in Figure 2. This mechanism divides the bids of the agents to 
classes according to the figures’ geometric area, runs a l-greedy algorithm in each class, 
and allocates the figures to agents in the class that achieved the highest result. From the 
algorithmic aspect, this is the first algorithm for packing weighted convex figures that 
we know of, and it achieves an O(R)-approximation. We use this algorithm to construct 
a polynomial-time and IC mechanism with the same approximation ratio for the social 
welfare. The payments are exactly the “critical-values” for the agents, 1.e., the minimal 
declaration for which they still win the auction. 


l1 Note that an agent can manipulate the values of & by affecting R and Q. Therefore, for incentive 
compatibility, the mechanism is assumed to know the true values of R and Q. 

12 This approximation is exponentially better than the approximation ratio we achieve for this 
problem in the USM model and than the ratio we achieve for general convex figures in the 
KSM model. 
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The Classes-by-Area 1-Greedy (CBA-1G) Mechanism: 
Allocation: 

Step 1: Divide the given input to m = 2log(R) classes according to their area. 

A figure s belongs to class c if g(s) € [W?-2°,W* -2°+!) (for c € {0,...,m—1}). 
Step 2: Perform the l-greedy algorithm per each class. Denote the welfare achieved 
by class c by V°. 

Step 3: Output the allocation in the class c for which the 1-greedy algorithm achieved 











TERE 





Payments: 
Denote the winning class as class 1, and the class with the second-highest welfare as 
class 2. Let V1 r= Vv! — v;, and let j be the first figure that intersects figure i and wins, 

when we run the greedy algorithm where agent i is removed. Let z; be “gy if 

such j exists, and 0 otherwise. 

A winning agent i pays: P(i) = max{V? — Yz, , Zi}, and any losing agent pays 0. 


Fig. 2. A mechanism for selling arbitrary convex figures. This mechanism is incentive compatible 
in the KSM model and achieves an O(R)-approximation for the social welfare 


The payments in the CBA-1G mechanism are chosen as follows: to win the auction, 
each agent should be both a winner in her class and her class should beat all other 
classes. Bidding above the value z; in the mechanism’s description, is a necessary and 
sufficient condition for agent i to win in her class. However, if agent i bids below V? ~ 
vy} ; and still wins in her class, her class will definitely lose. 


Theorem 10. When the agents bid for compact convex figures in R* with an aspect 
ratio R, the CBA-1G mechanism achieves an O(R)-approximation. This mechanism is 
IR and IC for the KSM model”, and runs in polynomial time. 


Proof sketch: We show that the approximation ratio achieved in each class is O(R-), 
where Re is the aspect ratio'* in class c. Due to a general proposition we prove, the ap- 
proximation ratio achieved by choosing the best class is O(3;, Re). Finally, we show that 
>Re = O(R), by dividing this sum to two geometric series. For proving IC, we show 
that the given payments are indeed the critical values for the agents, i.e. the smallest 
declarations for which they still win. O 


4.2 Mechanisms for Axis-Aligned Rectangles 


In the full version of this paper ([2]) we present an allocation algorithm, called the 
Shifting Algorithm, which is based on an algorithm by Khanna et al.([8]) with some mi- 
nor changes. They studied a model where axis-aligned rectangles lie in an n x n array, 
and they proved an O(/og(n))-approximation for the weighted packing problem. This 
approximation is the best approximation currently known for weighted packing of axis- 
parallel rectangles. Our algorithm gives an O(/og(R))-approximation in a slightly more 
general model where the rectangles can lie in any axis-parallel location in the plane. By 


B We observe that this mechanism is not IC in the USM model. 
4 Le. the ratio between the maximal diameter and the minimal width of figures in this class. 
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carefully defining a payment scheme, we use this allocation rule for designing an IC 
polynomial-time mechanism achieving an O(/og(R))-approximation for the social wel- 
fare. We call this mechanism the Shifting Mechanism and we summarize its properties 


in the following theorem”: 


Theorem 11. When the agents bid for axis-parallel rectangles in RÊ? with an aspect 
ratio R, the Shifting Mechanism achieves an O(log(R))-approximation. This mecha- 
nism is individually rational and incentive compatible for the KSM model, and runs in 
polynomial time. 


5 Conclusion and Further Research 


In this paper, we study auctions in which the agents bid for convex figures in the plane. 
We present mechanisms that run in polynomial time, in which the selfish players’ best 
strategy is to send their true private data to the auctioneer. We suggest using the aspect 
ratio R, which measures how diverse are the dimensions of the figures, for analyzing 
the economic efficiency of the mechanisms. 

In the KSM model, we were able to achieve the best approximation currently known 
for weighted axis-parallel rectangle packing (log(R)) in an IC mechanism. Lehmann et 
al. [10] showed that the best polynomial-time approximation for combinatorial auctions 
(for single minded bidders) can be achieved with an IC mechanism. On the other hand, 
recent results showed settings in which the optimal algorithmic approximation ratio 
cannot be achieved by IC mechanisms (see, e.g., [1,9]). Whether such gap exists in our 
model is an interesting open question: 


Open Problem: Can the best polynomial-time approximation schemes for packing 
convex figures (general figures, rectangles, or axis-parallel rectangles) be implemented 
by incentive-compatible mechanisms ? 

The mechanism for combinatorial auctions presented in [10] achieves the same ap- 
proximation both for the USM model and the KSM model. Our results might indicate 
that, in our model, a gap exists between the approximation achievable in both informa- 
tion models. For general convex figures, the approximation we achieve in the KSM and 


the USM models are O(R) and O(R a respectively. For axis-parallel rectangles, the 
gap in our results is even exponential. 


Open Problem: /n settings where the agents are “single minded”, is there a gap be- 
tween the best approximation achievable in the KSM and in the USM models? 

We present some novel algorithmic results regarding packing of convex figures and 
arbitrary rectangles. We have not been able to show that these results are tight. 


Open Problem: Js there an o(R)-approximation scheme for packing general convex 
figures, or evenfor packing rectangles (not necessarily axis-parallel) ? 

Our results may also be useful in deriving approximation results for the problem of 
packing weighted convex bodies in dimensions higher than two’°. 


I5 An easy observation is that the Shifting Mechanism is not IC in the USM model. We also 
note that the Shifting Mechanism achieves an Q(R)-approximation for general rectangles (not 
necessarily axis-parallel). 

16 However, the economic interpretation of such auctions is not always clear. 
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Abstract. In this paper we investigate the computational complexities 
of a combinatorial problem that arises in the reverse engineering of pro- 
tein and gene networks. Our contributions are as follows: 





— We abstract a combinatorial version of the problem and observe 
that this is “equivalent” to the set multicover problem when the 
“coverage” factor k is a function of the number of elements n of the 
universe. An important special case for our application is the case 
in which k = n—1. 

— We observe that the standard greedy algorithm produces an approx- 
imation ratio of (log n) even if k is “large” i.e. k = n — c for some 
constant c > 0. 

— Let 1 < a < n denotes the maximum number of elements in any 
given set in our set multicover problem. Then, we show that a non- 
trivial analysis of a simple randomized polynomial-time approxima- 
tion algorithm for this problem yields an expected approximation 
ratio E[r(a,k)] that is an increasing function of a/k. The behavior 
of E[r(a,k)| is “roughly” as follows: it is about In(a/k) when a/k is 
at least about e° 7.39, and for smaller values of a/k it decreases to- 
wards 2 exponentially with increasing k with lim, ~~ 0 E[r(a, k)| < 2. 
Our randomized algorithm is a cascade of a deterministic and a ran- 
domized rounding step parameterized by a quantity @ followed by a 
greedy solution for the remaining problem. 


1 Introduction 


Let [x,y] is the set {2,2+1,2+2,...,y} forintegers x and y. The set multicover 
problem is a well-known combinatorial problem that can be defined as follows. 
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Problem Name: SCx. 

Instance <7n,m,k >: An universe U = [1,n], sets $1,S2,...,5m CU 
with U™,S; = U and a “coverage factor” (positive integer) k. 

Valid Solutions: A subset of indices J C [1,m] such that, for every 
element z€U,|jel:2€S;|>k. 

Objective: Minimize |T|. 


SC; is simply called the Set Cover problem and denoted by SC; we will 
denote an instance of SC simply by <n,m> instead of <n,m,1>. 

Both SC and SC, are already well-known in the realm of design and analysis 
of combinatorial algorithms (e.g., see [14]). Let 3 < a < n denote the maximum 
number of elements in any set,i.e., a = MaXiefi,m]{|S:|}. We summarize some of 
the known relevant results for them below. 


Fact 1 

(a) [4] Assuming NP Z DTIME(n'8€") | instances < n,m > of the SC 
problem cannot be approximated to within a factor of (1 —e) lnn for any constant 
0 <é< 1 in polynomial time. 

(b) [14] An instance <n,m,k> of the SCx problem can be 

(1+]ln a)-approrimated in O(nmk) time by a simple greedy heuristic that, at every 
step, selects a new set that covers the maximum number of those elements that 
has not been covered at least k times yet. It is also possible to design randomized 
approximation algorithms with similar expected approximation ratios. 


1.1 Summary of Results 


The combinatorial problems investigated in this paper that arise out of reverse 
engineering of gene and protein networks can be shown to be equivalent to SC; 
when k is a function of n. One case that is of significant interest is when k is 
“large”,i.e., k = n — c for some constant c > 0, but the case of non-constant c is 
also interesting (cf. Questions (Q1) and (Q2) in Section 2). Our contributions 
in this paper are as follows: 


— In Section 2 we discuss the combinatorial problems (Questions (Q1) and 
(Q2)) with their biological motivations that are of relevance to the reverse 
engineering of protein and gene networks. We then observe, in Section 2.3, 
using a standard duality that these problems are indeed equivalent to SC, 
for appropriate values of k. 

— In Lemma 1 in Section 3.1, we observe that the standard greedy algorithm 
SC; produces an approximation ratio of (logn) even if k is “large”, ie. 
k =n — c for some constant c > 0. 

— Let 1 < a < n denotes the maximum number of elements in any given set 
in our set multicover problem. In Theorem 2 in Section 3.2, we show that a 
non-trivial analysis of a simple randomized polynomial-time approximation 
algorithm for this problem yields an expected approximation ratio E[r(a, k)| 
that is an increasing function of a/k. The behavior of E[r(a, k)] is “roughly” 
as follows: it is about In(a/k) when a/k is at least about e? ~ 7.39, and for 
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smaller values of a/k it decreases towards 2 exponentially with increasing k 
with lim, /~.0 E[r(a, k)| < 2. More precisely, E[r(a, k)] is at most 


1+ Ina, ifk=1 

(1 +e7~-)/5) In(a/(k — 1)), if a/(k —1) >e* andk>1 
min{2+2-e7*-D/5 24 (e7? + e7 9/8) . 2} 

~ min{2+2-e*-D/5 2 +0.46. 2} if a/(k—1)<e?andk>1 


Some proofs are omitted due to lack of space. 


1.2 Summary of Analysis Techniques 


— To prove Lemma 1, we generalize the approach in Johnson’s paper [6]. A 
straightforward replication of the sets will not work because of the depen- 
dence of k on n, but allowing the “misleading” sets to be somewhat larger 
than the “correct” sets allows a similar approach to go through at the ex- 
pense of a diminished constant. 

— Our randomized algorithm in Theorem 2 is a cascade of a deterministic and 
a randomized rounding step parameterized by a quantity @ followed by a 
greedy solution for the remaining problem. 

— Our analysis of the randomized algorithm in Theorem 2 uses an amortized 
analysis of the interaction between the deterministic and randomized round- 
ing steps with the greedy step. For tight analysis, we found that the standard 
Chernoff bounds such as in [1,2,10,14] were not always sufficient and hence 
we had to devise more appropriate bounds for certain parameter ranges. 


2 Motivations 


In this section is to define a computational problem that arises in the context of 
experimental design for reverse engineering of protein and gene networks. We will 
first pose the problem in linear algebra terms, and then recast it as a combina- 
torial question. After that, we will discuss its motivations from systems biology. 
Finally, we will provide a precise definition of the combinatorial problems and 
point out its equivalence to the set multicover problem via a standard duality. 

Our problem is described in terms of two matrices A € R”*” and B e R"™*™ 
such that: 


— Ais unknown; 

— B is initially unknown, but each of its columns, denoted as B1, Bo,..., Bm, 
can be retrieved with a unit-cost query; 

~— the columns of B are in general position, i.e., each subset of k < n columns 
of B is linearly independent; 

~ the zero structure of the matrix C = AB = (c;) is known, ie., a binary 
matrix C° = (c?,) € {0,1}"*™ is given, and it is known that cij = 0 for 
each i, j for which c}; = 0. 
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The objective, “roughly speaking”, is to obtain as much information as possible 
about A (which, in the motivating application, describes regulatory interactions 
among genes and/or proteins), while performing “few” queries (each of which 
may represent the measuring of a complete pattern of gene expression, done 
under a different set of experimental conditions). 

Notice that there are intrinsic limits to what can be accomplished: if we 
multiply each row of A by some nonzero number, then the zero structure of C is 
unchanged. Thus, the best that we can hope for is to identify the rows of A up 
to scalings (in abstract mathematical terms, as elements of the projective space 
P”~'). To better understand these geometric constraints, let us reformulate the 
problem as follows. Let A; denote the it” row of A. Then the specification of 
C? amounts to the specification of orthogonality relations A;-B; = 0 for each 
pair i,j for which ce. = 0. Suppose that we decide to query the columns of B 
indexed by J = {ji,..., je}. Then, the information obtained about A may be 
summarized as A; € Hja where “L” indicates orthogonal complement, and 


Hai = span {B;, j € Ji}, 
J, ={j |j €J and cj, =0}. (1) 


Suppose now that the set of indices of selected queries J has the property: 
each set J;,2 = 1,... n, has cardinality > n — k, (2) 


for some given integer k. Then, because of the general position assumption, the 
space Hz; has dimension > n — k, and hence the space Hii has dimension at 
most k. 

The most desirable special case is that in which k = 1. Then dim H7, < 1, 
hence each A; is uniquely determined up to a scalar multiple, which is the best 
that could be theoretically achieved. Often, in fact, finding the sign pattern 
(such as “(+,+,-,0,0,-,...)”) for each row of A is the main experimental 
goal (this would correspond, in our motivating application, to determining if 
the regulatory interactions affecting each given gene or protein are inhibitory or 
catalytic). Assuming that the degenerate case Hz; = {0} does not hold (which 
would determine A; = 0), once that an arbitrary nonzero element v in the 
line Hy; has been picked, there are only two sign patterns possible for A; (the 
pattern of v and that of —v). If, in addition, one knows at least one nonzero sign 
in A;, then the sign structure of the whole row has been uniquely determined 
(in the motivating biological question, typically one such sign is indeed known; 
for example, the diagonal elements ai;, 1.e. the ith element of each A;, is known 
to be negative, as it represents a degradation rate). Thus, we will be interested 
in this question: 


find J of minimal cardinality such that |J;| > n— 1, i =1,...,n. (Q1) 


If queries have variable unit costs (different experiments have a different associ- 
ated cost), this problem must be modified to that of minimizing a suitable linear 
combination of costs, instead of the number of queries. 
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More generally, suppose that the queries that we performed satisfy (2), with 
k > 1 but small k. It is not true anymore that there are only two possible sign 
patterns for any given A;, but the number of possibilities is still very small. For 
simplicity, let us assume that we know that no entry of A; is zero (if this is 
not the case, the number of possibilities may increase, but the argument is very 
similar). We wish to prove that the possible number of signs is much smaller 
than 2”. Indeed, suppose that the queries have been performed, and that we 
then calculate, based on the obtained B,’s, a basis {v1,...,vx%} of Hz, (assume 
dim HF; = k; otherwise pick a smaller k). Thus, the vector A; is known to 
k 
have the form >D ÀrUr for some (unknown) real numbers À1,..., Ag. We may 
r=1 
assume that A; Æ 0 (since, if A; = yaar Arvr, the vector €v + eee Nees 
with small enough €, has the same sign pattern as A;, and we are counting 
the possible sign patterns). If Ay > 0, we may divide by A; and simply count 
how many sign patterns there are when A; = 1; we then double this estimate 
to include the case A; < 0. Let up = col(vi;,...,Unr), for each r = 1,...,k. 
Since no coordinate of A; is zero, we know that A; belongs to the set C = 


RETEN (L U ses U Ln) where, for each 1 < s < n, L, is the hyperplane in R*—! 


consisting of all those vectors (A2,..., Ax) such that D ArVer = ~Vs1. On 
each connected component of C, signs patterns are constant. Thus the possible 
number of sign patterns is upper bounded by the maximum possible number of 
connected regions determined by n hyperplanes in dimension k—1. A result of L. 
Schlafli (see [3,11], and also [12] for a discussion, proof, and relations to Vapnik- 
Chervonenkis dimension) states that this number is bounded above by (n, k — 
1), provided that k—1 < n, where (n, d) is the number of possible subsets of an 


d d 
n-element set with at most d elements, that is, (n, d) = S e < 25 < 
i=0 ' 
d 
(=) . Doubling the estimate to include A; < 0, we have the upper bound 
2(n, k-1). For example, (n, 0) = 1, &(n, 1) = n+1, and O(n, 2) = 5(n?+n42). 
Thus we have an estimate of 2 sign patterns when k = 1 (as obtained earlier), 
2n+2when k = 2, n? +n +2 when k = 3, and so forth. In general, the number 
grows only polynomially in n (forfixed k). 

These considerations lead us to formulating the generalized problem, for each 
fixed k: find J of minimal cardinality such that \J;\ > n — k for all i =1,...,n. 
Recalling the definition (1) of J;, we see that J; = J(\T;, where T; = {7 | ch, = 
0}. Thus, we can reformulate our question purely combinatorially, as a more 
general version of Question (Q1) as follows. Given sets 


lC ilera mie ta henn: 
and an integer k < n, the problem is: 


find J C {1,..., m} of minimal cardinality such that |J NT:| 2 n — k, 
L<i<n. (Q2) 
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For example, suppose that k = 1, and pick the matrix C® € {0,1}"*" in such a 
way that the columns of C® are the binary vectors representing all the (n—1)- 
element subsets of {1,...,n} (so m = n); in this case, the set J must equal 
{1,...,m} and hence has cardinality n. On the other hand, also with k = 1, 
if we pick the matrix C® in such a way that the columns of C® are the binary 


vectors representing all the 2-element subsets of {1,...,n} (so m = n(n —1)/2), 
then J must again be the set of all columns (because, since there are only two 
zeros in each column, there can only be a total of 2@ zeros, 2 = |J|, in the 


submatrix indexed by J, but we also have that 2@ > n(n — 1), since each of the 


nrows must have > n — 1 zeros); thus in this case the minimal cardinality is 
n(n — 1)/2. 


2.1 Motivations from Systems Biology 


This problem was motivated by the setup for reverse-engineering of protein and 
gene networks described in [8,9] and reviewed in [13]. We assume that the time 
evolution of a vector of state variables x(t) = (21(t),...,2,(t)) is described by 
a system of differential equations: 


Tı = fili -3 n, Pl,- Pm) 
T2 = faltı,. -3 Tns Pl; Pm) 


A T E 2 ey 2 


(in vector form, “t = f(x, py’), where p = (p1,..., Pm) is a vector of parame- 
ters, representing for instance the concentrations of certain enzymes which are 
maintained at a constant value during a particular experiment. There is a ref- 
erence value p of p, which represents “wild type” (that is, normal) conditions, 
and a corresponding steady state z. That is, f(%,p) = 0. We are interested 
in obtaining information about the Jacobian of the vector field f evaluated at 
(Z,p), or at least about the signs of the derivatives Of;/0x,;(Z, p). For example, 
if Of,/0z; > 0, this means that z; has a positive (catalytic) effect upon the rate 
of formation of z;. The critical assumption, indeed the main point of [8,9], is 
that, while we do not know the form of f, we do know that certain parameters 
p; do not directly affect certain variables x;. This amounts to a priori biolog- 
ical knowledge of specificity of enzymes and similar data. This knowledge will 
be summarized by the binary matrix C° = (c?,) € {0,1}"*™, where “ch = 0” 
means that p; does not appear in the equation for z;, that is, Of;/Op; = 0. 
The experimental protocol allows us to make small perturbations in which 
we change one of the parameters, say the kth one, while leaving the remaining 
ones constant. (A generalization would allow for the simultaneous perturbation 
of more than one parameter.) For the perturbed vector p ~ p, we measure the 
resulting steady state vector x = &(p). (Mathematically, we suppose that for 
each vector of parameters p in a neighborhood of p there is a unique steady 
state €(p) of the system, where € is a differentiable function. In practice, each 


Randomized Approximation Algorithms for Set Multicover Problems 45 


such perturbation experiment involves letting the system relax to steady state, 
and the use of some biological reporting mechanism, such as microarrays, in 
order to measure the expression profile of the variables x;.) For each of the 
possible m experiments, in which a given p; is perturbed, we may estimate the 
n “sensitivities” 

_ $i 1 


bij = pP) S- i(P + pjej) —&(p)), t=1,...,n 
j ap, P By — p; (fi( + pje;j) — &:(D)) 








(where e; € R” is the jth canonical basis vector). We let B denote the matrix 
consisting of the b;;’s. (See [8,9] for a discussion of the fact that division by 
Pj — Pj, which is undesirable numerically, is not in fact necessary.) Finally, we let 
A be the Jacobian matrix Of /Oz and let C be the negative of the Jacobian matrix 
Of /Op. From f(E(p), p) = 0, taking derivatives with respect to p, and using the 
chain rule, we get that A = BC. This brings us to the problem stated in this 
paper. (The general position assumption is reasonable, since we are dealing with 
experimental data.) 


2.2 Combinatorial Formulation of Questions (Q1) and (Q2) 


Problem Name: CP, (the k-Covering problem that captures Ques- 
tion (Q1) and (Q2))' 

Instance < m,n,k >: U = [l,m] and sets Ti,Tz,..., Tn C U with 
Uy =U, 

Valid Solutions: A subset U’ C U such that |U'NT;| > n— k for each 
i efl, n]. 

Objective: Minimize \U"|. 


2.3 Equivalence of CP; and SC,,_~ 


We can establish a 1-1 correspondence between an instance < m,n, k > of CP; 
and an instance <n, m,n — k> of SC,_, by defining S; = { 7 | i € Tj} for each 
i € {1, m]. It is easy to verify that U” is a solution to the instance of CP, if and 
only if the collection of sets Su foreach u € U” is a solution to the instance of 
SC ik: 


3 Approximation Algorithms for SC; 


An €-approximate solution (or simply an €-approximation) of a minimization 
problem is defined to be a solution with an objective value no larger than € times 
the value of the optimum. It is not difficult to see that SC; is NP-complete even 
when k = n — c for some constant ¢ > 0. 


' CP,-1 is known as the hitting set problem [5, p. 222]. 
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3.1 Analysis of Greedy Heuristic for SC, for Large k 


Johnson [6] provides an example in which the greedy heuristic for some instance 
of SC over n elements has an approximation ratio of at least log n. This ap- 
proach can be generalized to show the following result. 


Lemma 1. For any fixed c > 0, the greedy heuristic (as described in Fact 1(b)) 
5) loga n = N(logn) for 





has an approximation ratio of at least (4 — o(1)) (z= 


some instance <n, M, n — c> of SCrn—c. 


3.2 Randomized Approximation Algorithm for SC; 


As stated before, an instance <n, m, k > of SC, can be (1 + ln a)-approximated 
in O(mnk) time for any k where a = maxgegs{|5|}. In this section, we provide a 
randomized algorithm with an expected performance ratio better than (1 +lna) 
for larger k. Let S = {81, So,..., Sm}. 

Our algorithm presented below as well as our subsequent discussions and 
proofs are formulated with the help of the following vector notations: 


— All our vectors have m coordinates with the it coordinate indexed with the 
i set S; of S. 
— if V C S, then v € {0,1} is the characteristic vector of V, ie, vg, = 
Oif S: € V 
— 1 is the vector of all l’s, i.e. 1 = s; 
— S ={A€ S: i€ A} denotes the sets in S that contains a specific element 
H 
Consider the standard integer programming (IP) formulation of an instance < 
n,m,k> of SC, [14]: 


t>k for each 2 € U 
a € {0,1} for each AE S 


minimize lx subject to” 


A linear programming (LP) relaxation of the above formulation is obtained by 
replacing each constraint z4 € {0,1} by 0 < z4 < 1. The following randomized 
approximation algorithm for SC, can then be designed: 





1. Select an appropriate positive constant 8 > 1 in po following manner: 


Ina ikal 
PAINS —1)) ifa/(k-1)>e? andk>1 
otherwise 


2. Find a on x to the LP relaxation via any polynomial-time algorithm for 
solving linear programs (e.g. [7]). 
3. (deterministic rounding) Form a family of sets C° = {A € S: Bza > 1}. 
4. (randomized rounding) Form a family of sets Ct c S — C°by independent 
random choices such that Pr[A € C*] = Baa. 
5. (greedy selection) Form a family of sets C? as: 
while s*(c° +c’ +c*) < k for some i € U, insert to C? any A € S* — (UZ_9C"). 
6. Return C = C° UC! UC? as our solution. 
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Let r(a,k) denote the performance ratio of the above algorithm. 
Theorem 2.” E[r(a,k)| is at most 

1+dna, lie el 

(1 +e7&—Y/5) In(a/(k — 1)), ifa/(k —1)>e? andk > 1 


min{2 +2. e~ (k—1)/5 24 (e~? + e798) , a) 
~ min{ 2 +2. e7%-1)/5, 2 +0.46- Et ifa/(k-1)<e* andk>1 


Let OPT denote the minimum number of sets used by an optimal solution. 
Obviously, OPT> 1x and OPT> ne A proof of Theorem 2 follows by showing 
the following upper bounds on E/r(a,k)| and taking the best of these bounds 
for each value of a/k: 


1+Ina, ifk=1 

(1 +e7%-0/5) In(a/(k — 1)), if a/(k — 1) > e? andk > 1 
242-6 ne, ifa/(k-—1)<e* andk>1 
2+ (e7? +e79/8) . 2, ifa/(k-—1)<e? andk>1 


3.2.1 Proof of E[r(a,k)| <1+Inaif k=1, 
E[r(a,k)] < (1 +e7~®-/5) In(a/(k — 1)) ifa/(k — 1) > e? and k > 1, 
and E[r(a,k)] <2+2-e-*-)/5 ifa/(k—1) < e? andk>1 


For our analysis, we first define two following two vector notations: 


0 ra if Bra >1 ye 0 if 8ra >1 
“A ~ ) 0 otherwise A \ x, otherwise 


Note that c% = [x2] < 62%. Thus 12° < 1c? < 812°. Define bonus = pix?’ — 
1c. It is easy to see that E[1(c® + c!)] = Bla — bonus. 

The contribution of set A to bonus is Bx% — cù. This contribution to bonus 
can be distributed equally to the elements if A. Since |A| < a, an element 
i € [1, n] receives a total of at least b*/a of bonus, where b = s*(Bx° — c°) The 
random process that forms set C? has the following goal from the point of view 
of element i: pick at least g* sets that contain i, where g* = k — s‘c® These sets 
are obtained as successes in Poisson trials whose probabilities of success add 
to at least pê = 8(k — stx?). Let y* be random function denoting the number 
that element i contributes to the size of C?; thus, if in the random trials in 
Step 4 we found h sets from S* then yê = max{0,k — h}. Thus, E({r(a,k)] = 
E[1(2 + cl +c*)] < Ble + OL, Ely’ — ©] Let ¢ = zrs (e — 2°), We can 
parameterize the random process that forms the set C? from the point of view 
of element 2 asfollows: 


? The case of k = 1 was known before and included for the sake of completeness only. 
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— gt is the goal for the number of sets to be picked; 

— pt = B(k — stx?) = Bgt + (B — 1)q' is the sum of probabilities with which 
sets are picked; 

— bJa is the bonus of i, where bê = s*(@x° — c£) > (8 — 1)(k — gt — q’*); 

—~q >0,9° > 0 and g? + ¢ < k; 

— yt measures how much the goal is missed; | 

~ to bound E[r(a, k)] we need to bound E[y* — =}. 


3.2.1.1 g-Shortage Functions 


In this section we prove some inequalities needed to estimate E[y* — b] tightly. 
Assume that we have a random function X that is a sum of N independent 0-1 
random variables X;. Let E[X] = >>, Pr|X; = 1] = u and g < pu be a positive 
integer. We define g-shortage function as Y# = max{g — X,0}. Our goal is to 
estimate E[Y#]. 


Lemma 2. E[Y#] < e~” aa Ti pi. 


From now on we will assume the worst-case distribution of Y}, so we will 
assume that the above inequality in Lemma 2 is actually an equality (as it 
becomes so in the limit), i.e., we assume E[Y#] = e7” eis s— p’. For a fixed 


8,we will need to estimate the growth of E(Y2"] as a function of g. Let pg(@) = 
eP EJY 2P]. 


Lemma 3. p,(1) = Dino FI = gy 


Lemma 4. For f > 1, RA is a decreasing function of ß. 
g 


gf g 
Lemma 5. If g> 1 and p > 1 then aren, <er (47) 


+a 
Lemma 6. si <e - e7401—1/2) 


3.2.1.2 Putting All the Pieces Together 


In this section we put all the pieces together from the previous two subsections 
to prove our claim on E[r(a,k)]. We assume that 6 > 2 if k > 1. Because we 
perform analysis from the point of view of a fixed element 7, we will skip 2 as a 
Seay as appropriate. As we observed in Section 3.2.1, we need to estimate 
Bly — 2] and b > (8 —1)(k — g — q). We will also use the notations p and q as 
defined there. 
Obviously if g = 0 then y = 0. We omit the case of k = 1 and assume that 
k > 1 for the rest of this section. We first consider the “base” case of g = 1 and 
q = 0.Since q = 0, cœ? = x°. Thus, b = s*(Bc? — 2) = (8—1)s*c® = (B—1)(k—1). 
Next, we compute Ely]. Since p = Bg = 8, Ely] = E[Y,’] = e~*. 
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We postulate that 














a od i ae 
= eh e kr! 
—-1l7~ a 
=ef(6-1)> > l 
=6+m(8-1) 2n (3) 


It is easy to see that, for the base case, Ef1(c® + c! + c*)] < Bla < In(a/(k — 
1))OPT. 

Now we consider the “non-base” case when either g > 1 or g > 0. Compared 
to the base case, in a non-base case we have bonus b decreased by at least 


(B — 1)(g +q — 1)/a. Also, Ely) = E[Y}] = E[y7st PP), 


Bly fete-Day 


< e—(9tq—1)/5° 
EY) ] -7 


Lemma 7. 


Summarizing, when bonus is decreased by at most (8 — 1)(g +q — 1)/a = 
(8—1)t/a, we decrease the estimate of Efy] by multiplying it with at least e~*/®. 
As a function of t = g +q — 1 we have 


Epl- taset Egiya E (es sate ra 


This is a convex function of t, so its maximal value must occur at one of 
the ends of its range. When t = 0 we have 0, and when t = k — 1 we have 
(BME eg (k-1)/ 5. As a result, our expected performance ratio for k > 1 is 
given by 


E(r(a,k)] < Ale + YL, Ely‘ — Ë] 

< BOPT + t e-(k-1)/5 

< B(1 + e~@-D/)OPT 

z (1 + e~ *-)/5) In(a/(k — 1)) OPT if a/(k — 1) > e? 

~ \2-(14+e7%-0/5) OPT if a/(k — 1) < €? 
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Abstract. Given an undirected n-node graph and a set C of m cuts, the mini- 
mum crossing spanning tree is a spanning tree which minimizes the maximum 
crossing of any cut in C, where the crossing of a cut is the number of edges in the 
intersection of this cut and the tree. This problem finds applications in fields as 
diverse as Computational Biology and IP Routing Table Minimization. 

We show that a greedy algorithm gives an O(r log n} approximation for the prob- 
lem where any edge occurs in at most r cuts. We then demonstrate that the prob- 
lem remains NP-hard even when G is complete. For the latter case, we design 
a randomized algorithm that gives a tree T with crossing O((log m + logn) - 
(OPT + log n)) w.h.p., where OPT is the minimum crossing of any tree. 

Our greedy analysis extends the traditional one used for set cover. The random- 
ized algorithm rounds a LP relaxation of a corresponding subproblem in stages. 


1 Introduction 


Given a graph G = (V,E) with n nodes and a family of cuts C = {C1,..., Cm}, the 
minimum crossing tree is a spanning tree T, which minimizes the maximum crossing of 
any cut, where the crossing ofacut C; isdefinedas |F(T)NC;|. Ifthe family of cuts is C 
={(v,V\v):v € V}, then the MCST problem reduces to finding the minimum degree 
spanning tree problem which has been widely studied [8]. Hence, NP-completeness of 
the minimum degree spanning tree problem [7] shows that MCST problem is NP-hard. 

In this paper, we show approximation guarantees for the greedy algorithm for the 
MCST problem. 


Theorem 1. Given a graph G = (V, E) and a family of m cuts C={C),...,Cm}, a 
greedy algorithmfor MCSTproblem gives a spanning tree T which crosses any cut in 
C at most O(r - logn) times the maximum crossing ofan optimal tree. 


Although the minimum degree spanning tree problem is trivial on complete graphs, 
surprisingly, the MCST problem remains difficult even for this special case. We show 
that the decision version of even this version of the MCST problem is NP-complete. 


Theorem 2. Given a complete graph G, set ofcuts C and a positive integer k, the 
problem ofdetermining whether there exists a spanning tree of G which crosses any cut 
in C at most k times, is NP-complete. 


“ Supported in part by NSF grant CCR-0105548 and ITR grant CCR-0122581 (The ALADDIN 
project). 
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A proof of the above theorem appears in the Appendix. The particular case of com- 
plete graphs finds application in fields as varied as IP routing and computational bi- 
ology. We give improved algorithm for the MCST problem on complete graph which 
gives better performance guarantees. 


Theorem 3. There is a randomized LP rounding based algorithm, which given a com- 
plete graph G and a family ofcuts C={Cy,...,Cm}, outputs a spanning tree T such 
that crossing for any cut C; € C is O(log m + logn) - (OPT + logn)), where OPT is 
the maximum crossing ofan optimal tree. 


1.1 Motivation: Chimerism in Physical Mapping 


The MCST problem finds important applications in computational biology. The physi- 
cal mapping problem of the human genome project is to reconstruct the relative position 
of fragments of DNA along the genome from information on their pairwise overlap. One 
has a collection of clones and a set of short genomic inserts (called probes). A probe 
defines a single location where a given subset of clones coincide. For each probe/clone 
pair, it can be determined whether the clone contains the probe as a subsequence using 
biological techniques. The problem is to construct the order in which the probes would 
occur along the original chromosome that is consistent with the given the probe/clone 
incidence matrix. This can be done efficiently if there is no chimerism. Chimerism is the 
result of concatenating two or more clone from different parts of the genome, produc- 
ing a chimeric clone -one that is no longer a simple substring of the chromosome. More 
formally, the problem is as follows: Given a probe-clone incidence matrix A, with rows 
indexed by probes and columns by clones, and the entry a;; is 1 iff probe 7 occurs in 
clone 7. If there is no chimerism, then the problem is reduced to finding a permutation 
of rows so that ones in each column are consecutive (called as 1-C1P) and this can be 
solved efficiently in polynomial time [1]. However, in the presence of chimerism, the 
problem is more difficult. Then, we need to find a permutation 7 of rows, such that 
each column has at most k blocks of consecutive ones (called as k-consecutive ones 
property or k-C1P), if the chimeric clones are a concatenation of at most k clones. The 
decision version of this problem i.e “Does a given 0-1 matrix have the k-consecutive 
ones property ?” has been proven to be NP-complete for k > 2in [5]. 


1.2 k-C1P and Vector TSPs 


A classical way to solve the k-C1P problem is to reduce it to a particular multidimen- 
sional TSP problem called the Vector TSP (vISP). This problem is defined on a com- 
plete graph G = (V,E), where each edge e € E is assigned an m-dimensional cost 
c: E — {0,1}™. The cost of a tour T in G is the m-dimensional vector e(T) = 
ee E(T) c(e) and the objective is to minimize ||c(T)||oo.- 

The reduction from k-C1P to vTSP is straightforward. Each row of A becomes a 
node in G and the cost assigned to edge e = (2, 7) is set to the XOR-vector between the 
two rows a; and a,;. Now, let r be the permutation induced by a solution T of vISP, and 
let b(A”) be the maximum number of blocks of consecutive ones in A”. Then, we have 
that b(A") = lelles, Solving the vTSP problem is NP-hard by this reduction from the 
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2-C1P problem. However, since the Hamming distance obeys the triangle inequality, it 
is possible to use the standard Euler Tour short-cutting technique in order to compute a 
2r-approximate solution, given an r-approximation to the related Vector MST problem 
(vMST). 

The vMST can be formulated as the minimum crossing spanning tree problem on a 
complete graph G. Any column j of A can be seen as a cut C; = (V;, V \ V;) defined on 
G by setting V; = {v; € Vja:; = 0}. The cost of edge e = (i, 7) is as before the XOR- 
vector between a; and a; i.e. c(e) is a 0-1 vector, where the lt? entry corresponding to 
a cut C; is 1 iff the edge (i, 7) crosses C;. Here, the terminology that an edge e crosses 
a cut C is used interchangeably with e € C. For any tree T, lete(T) = ee ger ele). 


The ¿t? entry of the vector c(T’) is exactly the number of edges of T crossing the cut 
Ci. Thus, the minimum crossing spanning tree minimizes ||c(T)||,0. 


1.3 Motivation: IP Routing 


Another useful application of the MCST problem can be found in [2] where itis shown 
that the an efficient solution for the min-k-C1P can be used to obtain an good approx- 
imation for the Interval Routing problem: given a set of IP routing tables sharing the 
same host space, the problem is to reassign the IP addresses to the hosts in order to 
minimize the maximum size of any IP routing table. 

This IP routing table minimization problem, MIN-IP for short, can be formalized as 
follows. We are given a set R = {rj,...,7n} ofn routers and a set H = {hj,..., hm} 
of m destination hosts. Each router r; € R has a degree ĝ;, that is 6; out-edges, and a 
routing table specifying which of the out-edges to take for every host. The problem is 
to choose the IP addresses of the m hosts and construct the n IP routing tables so as to 
minimize the maximum size of a table, that is the maximum number of used entries in 
a table. 

In [2] it is shown that, given any r-approximation algorithm for the problem of 
determining a row permutation that minimizes the maximum number of blocks (of ones) 
in a boolean matrix A, an efficient 2r log m-approximation algorithmexistsfor MIN-IP, 
which exploits a matrix representation of the instances of the problem. 

Similar applications can be found also in designing interval routing schemes as 
proposed in [3, 4]. 


1.4 Related Work 


As observed earlier, the minimum degree spanning tree problem is a special case of the 
MCST problem. The best result for the minimum degree spanning tree problem are due 
to Furer and Raghavachari [8]. They construct a spanning tree with maximum degree 
at most A* + 1 where A* is the maximum degree of the optimal tree. The vMST 
problem has been considered by Greenberg and Istrail [6]. They give solution of cost 
O(s(A) -OPT + logn). Here s(A) = mazi<i<n ae aij. Note that r in Theorem 
1 is different from s(A) in [6]: r is the maximum number of cuts a given edge e can 
cross, where the cuts are defined by columns of A; s(A) is the sparsity of A i.e. the 
maximum number of 1 ’s in any row in A. Observe that r < 2 - s( A), but s(A) can be as 
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bad as m. Hence, our algorithm gives comparable or better performance guarantee than 
the algorithm in [6]. 

The paper is organized as follows. In Section 2, we describe a greedy algorithm for 
the MCST problem and prove Theorem 1. In Section 3, we give a randomized algorithm 
for the special case and prove the guarantees of Theorem 3. In the Appendix, we show 
that the MCST problem is NP-hard even for complete graphs. 


2 Greedy Algorithm for the General Case 


In this section, we show that the greedy algorithm gives an O(r - logn) approxima- 
tion for the MCST problem where r is defined as maxeeg HC € C: e € C}]. Given 
any subgraph H, the maximum number of times H crosses any cut in C is denoted by 
Cross(H,C). 


Greedy Algorithm: 
Feo 
while F is not a tree 
do 


Let e" be an edge which minimizes C'ross(F Ue,C) 
over all edges e € G which join two components of F’. 


Pe Fue! 





od 


First, we give a lower bound for the MCST problem. 


Lemma 1. GivenanyS C C, let k be the number of components formed after removing 
the edges from G of all cuts in S. Then 
so 
opt 2? nr 
|S] 

Proof. Any spanning tree of G must choose at least k—1 edges to join the k components 
formed after removing the edges of cuts in S. Each of these k — 1 edges crosses at least 
one of the cuts in S. Hence, the average crossing of such a cut in S is at least Sa 


The Proof of Theorem 1. Let the solution returned by the greedy algorithm be 7, and 
let? = Cross(T,,C). We can divide the running of the greedy algorithm in / phases. 
The i*” phase of the algorithm is the period when Cross(F,C) = i. Let k; denote the 
number of components in F when the it” phase ends. Let M; be the cuts which are 
crossed by 7 edges at the end of i” phase and m; = |M;l. 

Consider the running of the algorithm in the it? phase. The crossing number of 
at least m; cuts increases by 1 in the i” phase. Each edge can increase the crossing 
number of at most r cuts. Hence, in the i*” phase we must include at least | =] edges 
in F. Every edge, when included in F, reduces the number of components in F by 
exactly one. Therefore, we have the following inequality 


ki < ky-1 — ida (1) 
z 
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When the i*” phase ends, every edge joining two components of F must cross at least 
one of the cuts in Mj, else the greedy algorithm would choose such an edge in the i*” 
phase. Applying Lemma 1, we get the for each å, 

k;-1 


a 





opt > (2) 


Using (1) and (2), we have that for each 2, 


k; —1 
r x opt 





ki-i = k; > (3) 


Using k; > 2foreach 1 < l — land kj_, > kı, we have for each å, 


k: 
ki-1 — ki > IERT 
=> ki—1 > k; (1 + 


=> ko > kı(1 + 


Izpi) 
cn) 
As, ko = n and kı = 1,we get that 


n > (1 T Topi) 
=> logn > llog(1 i Irai) 


Using, log(1 + 2) > z — z and r * opt > 1 we get 


logn > (yall = Tea) 2 las 
=> | < 4rlogn x opt 


Hence, the greedy algorithm is a O(r log n) approximation. O 


3 A Randomized Algorithm for the Case of Complete Graphs 


In this section, we describe a randomized algorithm for MCST for complete graphs and 
prove that it gives a tree with maximum crossing O((log m + logn) - (OPT + log n)) 
with high probability, where n is the number of vertices in G and m is the number of 
cuts in C. 

The idea is the following: Start with each vertex as a different component and merge 
components in phases until a connected subgraph is obtained. In a phase, each compo- 
nent is represented by an arbitrarily chosen vertex of the component. We carefully se- 
lect some edges between the representative vertices by solving a multicommodity flow 
problem in each phase, so that the cuts in C are not crossed “too much”. We ensure that 
at least one edge is chosen out of each representative in every phase. Hence, the num- 
ber of components reduces by at least a factor of two and thus a connected subgraph is 
obtained in at most log, n phases. 

In phase p, we solve the following multicommodity flow problem on a graph G” 
constructed from a complete graph Gp (on the representative vertices in this phase) as 
follows. Let V (Gp) = {v1, v2,...,Un, } 


p . 


56 Vittorio Bilo et al. 


— For each undirected edge e = (u,v), add two directed edges ef = (u,v) and 
er = (v, u) in C, 

~ For each vertex v; € V (Gp) introduce a new vertex s,, in V(G’) and 

- Vv; € V(Gp), j £ i, add the directed edge (vj, sy,) in G”. 


Now, the flow problem on G” is the following. Each vertex v; € V(G’) is required 
to send a unit flow of commodity 7 to sy,. Let f1, fo,..., fn be the flows associated 
with each of the n commodities. Let f;(v) denote the net flow of it” commodity into 
the vertex v. The following integer program accomplishes our goal. 


min z 

st. z > Vecmianexe YCec 
Xe = i=1,....Np file) 

Vi = 1,..., Np 

fia) = Lewesen fiM u) — Laweney filv) Vo € V(G") 
fiw) = 0 Vu € V(G") \ {vi su, } 
fiv) =i 
fils, ) = —l 


file) € {0,1} Vee E(G’) 


We now describe the algorithm for the MCST problem. We will construct a con- 
nected subgraph H with a low maximum crossing. 


l. Initialize V(H) — V(G), E(H) — ¢,Go — G, Ro — V(G), p— 0. 
2. While H is not connected 
(a) Construct G’ from Gp. Solve the LP-relaxation of the corresponding integer 
program for phase p and obtain an integral solution X by randomized rounding 
of the optimum LP solution [10]. 
(b) Let BY = {e € Gp : Xe > 0}. F(A) — E(B) U F”. 
(c) p+ p + 1. Let Rp be the set of representative vertices (chosen arbitrarily one 
for each connected component of H), Gp is the complete graph on the vertices 
of Rp. 


Let T* be a optimal tree for the MCST problem and let OPT be the maximum cross- 
ing of any cut in T*. 


Proposition 1. Let z; be the optimum to the LP-relaxation in phase p. Then zp <S 
2OPT. 


Proof. We can construct a feasible solution of the LP from the optimum tree T* of 
value at most 2OPT. Let R; = {v1,...,Un,} be the set of representatives in phase i. 
From the Tree Pairing Lemma in [9], there exists a matching M between vertices of 
R; such that the paths in T* between the matched pairs of vertices are edge disjoint. 
We can use this matching to construct a feasible solution to the LP. Send a unit flow 
of commodity 7 on the directed path Py, 1, U (uj, sy,) and of commodity j on the path 
Py, vi U (Vi, Sv; ), where P(u, v) is the unique path in tree T* between matched pairs u 
and v. The above flow is a feasible flow as it satisfies all the flow constraints of the LP. 
Every edge of 7* carries at most two units of flow. Hence, the objective value z for this 


feasible flow, is at most 2OPT. Therefore, Zp < 20PT. 
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Proposition 2. [fan edge e = (u,v) crosses a cut C, then any other path between u 
and v also crosses the cut C at least once. 


Proof. If we remove all the edges in C from G, then u and v would be disconnected. 
Thus, every path from u to v contains an edge of C. 


We will use Proposition 2, to obtain and work with a special kind of optimum so- 
lution such that each flow path uses only two edges. Consider the flow decomposition 
for commodity 7 in the optimum solution of the LP-relaxation and consider a flow path 
P =< Uj, Viz, Vigs ++) Vips Sv; >. We can replace P by the path P’ =< Vi, Vips Sv; > 
without increasing the maximum crossing. From Observation 2, we know that any cut 
that the edge (v;i, vi, ) crosses will be crossed at least once by the path P. Therefore, 
P’ only reduces the number of crossings for the cuts in C and so we can replace P by 
P’. Thus, we can obtain a fractional optimum solution S* such that each flow path uses 
only two edges. This step greatly simplifies the subsequent analysis of the randomized 
rounding since every cut crosses every flow path at most once after this preprocessing. 


3.1 Rounding S* to an Integral Solution 


Let us describe the rounding of the fractional multicommodity flow obtained by solving 
the LP relaxation corresponding to phase p. The flow corresponding to each commodity 
is rounded independently of others. For each commodity i = 1,..., np, choose an edge 
e = (vi, vj) with probability f;(v;, vj). The corresponding flow is routed through the 
path < vi, Vj, sy, > and the edge (vi, v;) is included in the subgraph H. This is repeated 
for every commodity independently. 

In phase p, let the fractional optimum flow be f* and the optimum LP solution be 
z*. Let z(C) denote the number of edges crossing a cut C € C. Consider Y}, a 0-1 
variable associated with the jt? commodity, where 


1 if the integral flow crosses C 
Y; = , 
0 otherwise 


Therefore, 


e€E(Gp)NC 
2(C)= SY; 
j=l 


Elz(C)| = $521 Xece) f(e) 
=e Žec E(G:)NC Ži J(e) 
= Luec E(G:)NC Xe 

<% < 2-OPT 


z(C') is the sum of independent Bernoulli trials. Thus, we can use Chernoff bounds 
to bound the tail probability 
k2 B? 
= < E a REER 
Pr(|2(C) -EOI >k) < eeo) 
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Let 8 = E[z(C)] + logn and k = log, m + 2. Therefore, 


Pr(|2(C) — Elz(C)]| > kB) < exp(-E GEO ee) ) 
< exp(— (2log, rid) log) 
1 


az mn? 


Since E[z(C)] < 2OPT, we have that Pr(z(C) > (2(k + 1)OPT + klogn)) < 
z4 or Pr(z(C) > 2(log, m + 3) - OPT + (log, m + 2) -logn) < —, for any 
cut C € C in any phase p. We say that a “bad” event occurs in a phase p if some 
cut C € C has a high crossing in that phase. Thus, from the union bound we have 
Pr(bad event occurs in phase p) < 4. The algorithm has at most log, n phases. Thus, 


l 
Pr(”bad” event occurs in any phase) < -E (4) 





Thus, we have shown that in every phase the crossing of every cut is O((log,, m + 
3)OPT + (log, m + 2) - logn) with high probability. Hence, we obtain a solution of 
maximum crossing O((log,m + log.) - (OPT + log, n)) with probability at least 
Ca D 

Remark: For m > n, setting k = y (logam + 2) gives a slightly better solution 
with maximum crossing O(./log mlogn(OPT + logn)). 


4 Future Work 


We believe that better performance ratios can be obtained particularly for the MCST 
problem on complete graphs. Furthermore, more sophisticated methods than a simple 
greedy approach should be able to remove the factor of r in the general case. 
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Appendix: MCST for Complete Graphs Is NP-Hard 


In this section, we consider the MCST problem for complete graphs. We show that the 
problem is NP-hard even for this special case. In fact, we show that the decision version 
of the problem is NP-complete. 

Clearly, the decision problem is in NP. We reduce the 2-consecutive ones problem, 

2-C1P, to MCST. Given an x m matrix A, 2-C1P is the problem of determining whether 
there exists a permutation of rows such that in each column all ones occur in at most 2 
consecutive blocks. This problem has been shown to be NP-complete in [6]. 
Given any arbitrary n x m matrix A, make a complete graph G over n + 1 vertices, with 
one vertex corresponding to each row and anew dummy vertex s. For each column in A, 
include a cutin C naturally defined by the column: vertices with rows with 1 form one 
side of the cut. The dummy vertex s is always on the O-side of each cut. Also include in 
C singleton cuts, Cy = ({v}, V \ {v}) for every vertex in G. For each pair of vertices 
u and v, include in C the cut Cy, = ({u, v}, V \ {u, v}). Finally, let k = 4. 

We first show that if there exists a permutation of rows, 7, such that it has 2-Cl 
property, then there exists a spanning tree which crosses each cut in C at most four 
times. Consider the Hamiltonian path H which starts at s and then traverses the vertices 
in the order corresponding to permutation 7. Each cut corresponding to a column is 
crossed by the Hamiltonian path H exactly when the row permutation m switches from 
arow with 0 with arow with 1 or vice versa. As all the ones are in 2 consecutive blocks, 
each cut can be crossed at most four times. Introducing the dummy node corresponds 
to introducing a row with all zeros as the first row which clearly does not change 2-C1 
property. Also, a Hamiltonian path crosses each singleton cut at most two times and 
cut Cuy at most two times forany u,v € V. Hence, there exists a spanning tree which 
crosses every cut in C at most four times. 

Now, for the other direction we show that if there exists a spanning tree T which 
crosses every cutin C at most 4 times then there exists that a permutation m which has 
the 2-C1P property. We claim that any such tree must be a Hamiltonian path. As each 
singleton vertex is a cutin C, hence degree of each vertex is at most four. Suppose there 
exists a vertex u with degree four. For n > 5, there exists a vertex v which is not a 
neighbor of u. But, then the cut Cu» is crossed at least five times. Hence, all vertices 
have degree at most three. Suppose, for the sake of contradiction there exists a vertex 
u such that degr(u) = 3. Consider any vertex v which is not a neighbor of u. As T 
crosses Cy, at most four times, so degr(v) = 1. This implies that the total sum of 
degrees of nodes in T is at most3 * 4 + (n — 3). Hence, 2n — 2 < n +9 or equivalently, 
n < 11 which is a contradiction assuming larger n. Hence, every vertex must have 
degree at most two in T showing that T is a Hamiltonian path. 
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Let the hamiltonian path be (v3, ..., Uk, S, Uk+1, Vn). Consider the following per- 
mutation ofrows (Tk41,.- -Tn T1,- -», Tk) Where vi corresponds to row r; in the trans- 
formation. We claim that in each column, there cannot be more than two blocks of ones. 
Suppose for the sake of contradiction, there exists a column e; which has three blocks 
of ones. Thus, the cut corresponding to the Hamiltonian cycle formed by joining Vn 
and vı must cross the cut corresponding to column c; at least five times. But any cycle 
crosses any cut even number of times. Hence, it must cross the cut at least six times, but 
then the hamiltonian path must cross the cut at least five times, a contradiction. Hence, 
there exists a permutation which satisfies the 2-C1 property. This reduction shows that 
decision version of MCST problem for complete graphs is NP-complete. g 
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Abstract. We present a polynomial time 3/4-approximation algorithm 
for the maximum asymmetric TSP with weights zero and one. 

As applications, we get a 5/4-approximation algorithm for the (mini- 
mum) asymmetric TSP with weights one and two and a 3/4-approxi- 
mation algorithm for the Maximum Directed Path Packing Problem. 


1 Introduction 


Traveling salesperson problems with weights one and two have been studied 
for many years. They are an important special case of traveling salesperson 
problems with triangle inequality. Papadimitriou and Yannakakis [8] showed that 
the undirected minimization problem is MaxSNP-complete. On the other hand, 
they presented a 7/6-approximation algorithm with polynomial running time. 
Vishwanathan [9] considered the corresponding asymmetric problem ATSP(1, 2) 
and gave a 17/12-approximation algorithm. 

Let MaxATSP(O, 1) be the following problem: Given a directed complete loop- 
less graph with edge weights zero and one, compute a TSP tour of maximum 
weight. MaxATSP(O, 1) is a generalization of ATSP(1, 2) in the following sense: 
Vishwanathan [9] showed that any (1 — a@)-approximation algorithm for the for- 
mer problem transforms into an (1 + q@)-algorithm for the latter when replacing 
weight two with weight zero. (The other direction is not known to be true.) 

By computing a matching of maximum weight and patching the edges to- 
gether arbitrarily, one easily obtains a polynomial time 1/2-approximation al- 
gorithm for MaxATSP(O, 1). (Note that each edge has weight at least zero, thus 
we cannot loose any weight during the patching process.) Vishwanathan [9] was 
the first to improve on this by designing a 7/12-approximation algorithm with 
polynomial running time. In 1994, Kosaraju, Park, and Stein [6] gave a 48/63- 
approximation algorithm with polynomial time that also worked for maximum 
ATSP with arbitrary nonnegative weights. In their work, they also formulated 
the so-called path coloring lemma, which will be crucial for our algorithm. 
Blaser and Siebert [3] obtained a 4/3-approximation algorithm with running 
time O(n5/?) for ATSP(I, 2). This algorithm can also be modified to give a 2/3- 
approximation algorithm for MaxATSP(O, 1) with the same running time [4]. Fi- 
nally, Kaplan et al. [5] generalize this result by designing a 2/3-approximation 
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algorithm that works for maximum ATSP with arbitrary nonnegative weights 
but has a worse running time. 

Closely related to MaxATSP(O, 1) is the Directed Path Packing Problem DPP. 
Here we are given a directed graph G = (V, E). The aim is to find a subset P 
of node-disjoint paths of G such that the number of edges in P is maximized. 
By giving edges in G weight one and “non-edges” weight zero, any path packing 
transforms into a TSP tour by patching the paths arbitrarily together. On the 
other hand, any TSP tour yields a path packing by discarding all edges of weight 
zero. The only exception is the case where an optimum TSP tour has weight n. 
Here one weight one edge has to be discarded. 

Our main result is a 3/4-approximation algorithm for MaxATSP(O, 1) with 
polynomial running time. As corollaries, we get a 5/4-approximation algorithm 
for ATSP(1, 2) and a 3/4-approximation algorithm for DPP. 


1.1 Notations and Conventions 


For a set of nodes V, K(V) denotes the set of all edges (u,v) with u Æ v. In 
other words, (V, K(V)) is the complete loopless graph with node set V. 

For a multigraph H and an edge e of H, mz(e) € N denotes the multiplicity 
of e in H, that is, the number of copies of e that are present in H. If H is clear 
from the context, we will also omit the subscript H. 

A multigraph is called 2-path-colorable if its edges can be colored with two 
colors such that each color class is a collection of node-disjoint paths. (Double 
edges are not allowed in such a collection of paths.) 


1.2 Outline of Our Algorithm 


Kosaraju, Park, and Stein [6] formulate the so-called path coloring lemma. It 
states that if each node of a multigraph H has indegree and outdegree at most 
two and total degree at most three and H does not contain any 2-cycles (that 
is, a cycle with exactly two edges) or triple edges, then H is 2-path colorable. 
Kosaraju, Park, and Stein proceed with computing a cycle cover and a matching. 
(A cycle cover of a graph is a collection of node-disjoint directed cycles such that 
each node belongs to exactly one cycle.) If the matching is carefully chosen, then 
the union of the cycle cover and the matching fulfills the premises of the path 
coloring lemma and henceforth, is 2-path-colorable. (One also has to deal with 
the 2-cycles in the cycle cover separately, the interested reader is referred to the 
original work.) If one now takes the color class with the larger weight and patches 
the paths arbitrarily together, one gets a TSP tour that has at least half the 
weight of the combined weight of the cycle cover and the matching. The weight 
of an optimum cycle cover is at least the weight of an optimum TSP tour and the 
weight of an optimum matching is at least half the weight of an optimum TSP 
tour. Thus in the ideal case, this would yield an 3/4-approximation. However, 
Kosaraju, Park, and Stein have to deal with 2-cycles and have to avoid triple 
edges. Therefore, they only get a 48/63-approximation. This approach is refined 
in subsequent works [2, 7]. 
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In this work, we directly compute a maximum weight multigraph that fulfills 
the premises of the path coloring lemma. This is done via an LP approach 
(Section 3). The fractional solution H* is then rounded to an integer one via 
an iterated decomposition scheme (Section 4). Finally the integer solution is 
transformed into a TSP tour (or Path Packing) via the path coloring lemma. 


2 Admissible Multigraphs 


A directed multigraph is called admissible, if 


1. the indegree and outdegree of each node is at most two, 
2. the total degree of each node is at most three, 
3. between each pair of nodes, there are at most two edges (counted with mul- 
tiplicities). 
Let w : K(V) — {0,1} be a weight function. (This will be the weight function of 
the input of our algorithm for MaxATSP(Q, 1).) Our goal is to find an admissible 
multigraph H of maximum weight where the edges are weighted according to w, 
that is, each edge contributes weight my(e) - w(e). 
Lewenstein and Sviridenko [7], by reduction to the path coloring lemma of 
Kosaraju, Park, and Stein [6] (see Bläser [2] for a proof), show the following 
variant of the path coloring lemma: 


Lemma 1 (Path coloring lemma). If there are no 2-cycles on a cycle in an 
admissible multigraph G, then G is 2-path-colorable. 


Above, a 2-cycle on a cycle is a directed cycle v4,..., Uk, V1 with k > 3 such 
that (vj, vj-1) Gif i = 1, then i — lis k) is also an edge for some 7. 

We first compute an admissible multigraph of maximum weight. Then we 
have to remove 2-cycles on a cycle. In the case of weights zero and one, we are 
able to remove these 2-cycles without any loss of weight. 


3 LP for Maximum Weight Admissible Multigraphs 


For each edge e € K(V), there is a variable ze. If e = (u,v), we also write Tuy 
instead of xe. The following integral LP solves the problem of finding a maximum 
weight admissible multigraph: 


Maximize ` w(e)£e subject to 


e€ K(V) 

` Lay <2 for all v € V (indegree) 
UxV 

>» Diy ms 2 for all u € V (outdegree) 
vÆu 

D Tuv + So Zuu <3 forallueV (total degree) 
vu w#Ąu 

Pi to for all u,v E V, u £v (triple edge) 


Tuv E {0,1,2} for all u,v Ee V,u#v (multiplicity) 
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Constraints (outdegree) and (indegree) assure that the outdegree and indegree 
of each node are at most two. By (total degree), each node has total degree at 
most three. Constraint (triple edge) forbids more than two edges (counted with 
multiplicities) between each pair of nodes. The last condition ensures that each 
edge has integral multiplicity. By (triple edge), each edge can have multiplicity 
at most two. We now relax the integrality constraint: 


0< zw <2 foraluyveV,uFxv (multiplicity’) 


(Note that only the lower bound of (multiplicity’) is actually needed, the upper 
bound also follows from (triple edge).) Let xj, be an optimal solution of the 
relaxed LP. Let w* be its weight. We may assume that the z¥,, are rational num- 
bers. Their smallest common denominator D can be represented with poly(n) 
bits. We define the multigraph H* as follows. For all u and v with u Æ v, there 
are £uv' D copies of the edge (u,v) in H*. Furthermore, we define the bipartite 
multigraph B* with bipartition V UV’ (V’ is a copy of V) as follows: If there 
are d copies of the edge (u,v) in H*, then there are d copies of the edge (u, v’) in 
B*, The graph B* will be helpful in the decomposition of H*. By construction, 
H* and B* fulfill the following properties: 


(P1) The outdegree of each node v in H* is at most 2D. The degree of each 
node v € V in B* is at most 2D. 

(P2) The indegree of each node v in H* is at most 2D. The degree of each node 
v’ € V' in B* is at most 2D. 

(P3) The total degree of each node v in H* is at most 3D. The sum of the 
degrees of v and v’ in B* is at most 3D. 

(P4) Between each pair of distinct nodes u and v there are at most 2D edges 
in H*, For any pair of distinct nodes u and v, the sum of the number of 
edges between u and v’ and between v and u’ is at most 2D. 


4 A Rounding Procedure for H* 


We first assume that D = 2° is a power of two. We will deal with the general 
case later. We now decompose H* into two subgraphs Hj and Hš such that 
both Hý and Hš fulfill the properties (P1)—(P4) of the preceding section with 
D replaced by D/2. If we now replace H* with the heavier of Hf and HZ and 
proceed recursively, we will end up with an optimum admissible multigraph after 
log D = poly(n) such decomposition steps. 

In the following, H denotes the graph of the current iteration of the pro- 
cess outlined above and B is the corresponding bipartite graph. Our rounding 
procedure uses the fact that the edge weights are either zero or one. 

Alon [1] gives a similar procedure for edge-coloring bipartite multigraphs. 
This procedure is then used by Kaplan et al. [5] for decomposing a fractional 
solution of an LP for computing cycle covers. 
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4.1 Normalization of H 


We first ensure that H fulfills also the following property (P5) in addition to 
(P1)—(P4). 


(P5) For all u and v: If m(u,v) + m(v, u) = 2D, then m(u,v) and m(v, u) are 
both even. 


Algorithm 1 takes a multigraph H that fulfills (P1)—(P4) and transforms it 
into a multigraph H’ with the same weight that fulfills (P1)-(P5). 

The algorithm basically works as follows: Let u and v be two nodes. W.l.o.g. 
m(u,v) > m(v, u). Furthermore, assume that m(u,v) + m(v, u) = 2D and both 
m(u,v) and m(v, u) are odd. (As D is even, if one of m(u, v) and m(v, u) is odd, 
then both are.) Since D is a power of two, m(u,v) > D. 

If w(u, v) = 0, then we can simply remove all copies of (u,v). This does not 
change the weight of H. Furthermore, this does not affect (P1)—(P4), too. If 
w(v,u) = 0, then we remove all copies of (v, u). The interesting case is w(v, u) = 
w(u,v) = 1. Here we remove one copy of (u,v) and add one copy of (v, u). This 
does not change the weight. Since m(u,v) > D, m(u,v) > D thereafter. In 
particular, the indegree of u and the outdegree of v are both still at most 2D. 
Therefore, the resulting graph H’ fulfills (P1)—(P4), too. 

By construction, H” fulfills (P5). This shows the following lemma. 


Lemma 2. Given a multigraph H fulfilling (P1)-(P4) for some D = 2°, Algo- 
rithm I computes a multigraph H' that has the same weight as H and fulfills 
(PI)1}P5) for D. 


4.2 Decomposition of H 


Next we describe how to decompose a graph H fulfilling (P1)-(P5) for D into 
two graphs Hı and Hg fulfilling (P1)—(P4) for D/2. Let Hy be the heavier of Hı 
and Hə. We thereafter normalize Hı and proceed recursively. 

In the first for loop of Algorithm 2, we run through all edges (u,v). If the 
multiplicity of (u,v) is even in H, then we simply divide all the copies evenly 
between Hı and Ho. If the multiplicity is odd, then we divide all but one copy 
evenly between Hı and H2. Thereafter, all edges in H have multiplicity zero or 
one. 

If the indegree and outdegree of a node u in H are both odd, then we add 
the edge (u,u’) to B. Then we take two nodes of odd degree in a connected 
component of B. (If one such node exists, then there must exist another one.) 
We compute a path P connecting two such nodes and add the edges of P in 
an alternating way to Hı and Hg. If there are not any nodes of odd degree, 
each connected component of B is Eulerian. We compute Eulerian tours and 
distribute the edges in the same way as in the case of a path P. 

We claim that Hı and Hə fulfill (P1)-(P4) with parameter D/2. For (P1) 
note that we always remove the edges in pairs. If for instance an edge (u,v) is 
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Algorithm 1 Normalization of H 
Input: Multigraph H fulfilling (P1)-(P4) for D = 2° 
Output: Multigraph H” fulfilling w( H) = w(H") and (P1)-(P5) for D 
for all unordered pairs of nodes u,v € V, u Æ v do 
Let m be the multiplicity of (u,v) in H and m’ be the multiplicity of (v, u) in H. 
W.Lo.g. m >m. 
if m+m’ < 2D or m or m is even then 
insert (u,v) and (v,u) with multiplicities m and m’ into H’, respectively. 
else 
if w(u, v) = 0 and w(v,u) = 1 then 
insert m’ copies of (v, u) into H”. 
end if 
if w(u,v) = 1 and w(v,u) = 0 then 
insert m copies of (u,v) into H’. 
end if 
if w(u, v) = w(v,u) = 1 then 
insert m — 1 copies of (u,v) insert m’ + 1 copies of (v, u) into H” 
end if 
end if 
end for 


removed from H and added to Aj, then also an edge (u, w) is remove from H 
and added to Hy. In other words, the outdegree of u in Hı and H3 is always 
the same. The only exception is the case when the original outdegree of u is 
odd. Then one more edge is added to Hy, say, than to H2. However, since the 
outdegree in H was odd, it was at most D — 1. Therefore, the degree in A is at 
most D and in Hg, it is at most D — 1. The same argument works if the roles 
of Hı and Hə are interchanged. In the same way, we can show that H; and Hz 
fulfill (P2) with D/2. 

For (P3), we distinguish four cases, depending on the parity of the indegree 2 
and outdegree o of a node u in H. If both z and u are even, then the indegree and 
outdegree of u are 1/2 and o/2, respectively, in both Hı and Hz by construction. 
If both 7 and u are odd, then the indegree and outdegree are |i/2] and [o/2], 
respectively, in one of Hı and Ho, and [i/2] and |0/2} in the other of Hı and 
H3. This is due to the fact that we added the edge (u,u’) to B. In both cases, 
the indegree and outdegree sums up to (i + 0)/2 < 3D/2. Finally, we consider 
the case where either 7 or o is odd. We assume that 7 is odd, the other case is 
treated symmetrically. Then the indegree of u in Hı, say, is [¢/2] and in H2, it 
is [7/2]. In both subgraphs, the outdegree of u is 0/2. The total degree of u in 
H is however at most 3D — 1, since 3D is even. Therefore, i + 0 < 3D — 1 and 
[i/2| +0/2 < 3D/2. 

It remains to show that (P4) holds: Let m be the multiplicity of (u,v) in 
H and m’ be the multiplicity of (v,u) in H. If both m and m’ are even, then 
half of the copies of both (u,v) and (v, u) is added to A, and the other half are 
added to Hə in the first for loop. Thus the number of edges between u and v 
is (m+m')/2 < 2D/2 in both Hı and H2. If m is odd and m’ is even or vice 
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Algorithm 2 Decomposition of H 
Input: Multigraph H fulfilling (P1)-(P5) for some D = 2° 
Output: Multigraphs Hı and Hə fulfilling (P1)-(P4) for D/2 
for all (ordered) pairs of nodes u and v, u Æ v do 
Let m be the multiplicity of (u, v). 
Let h = |m/2|. 
Remove 2h copies of (u,v) from H. 
Add h copies to Hı and h copies to Ho. 
Update B accordingly. 
end for 
for all nodes u do 
if the indegree and outdegree of u in H are both odd then 
add (u, u’) to B 
end if 
end for 
while B contains nodes of odd degree do 
Choose two nodes a,b € V UV’ of odd degree in B that lie in the same connected 
component. 
Compute a path P from a to b. 
Remove the edges of P from H and add them to Hı and Hz in an alternating way. 
Skip all edges of the form (u,u’). Update B. 
end while 
for each connected component C of B do 
Compute a Eulerian tour of C. 
Remove the edges of C from H and add them to Hı and Ho in an alternating way, 
again skipping all edges of the form (u,w’). 
end for 


versa, then there are |(m + m’)/2| between u and v in both H, and H3 after 
the first for loop. Then one further copy is added to either Hı and H2. But since 
2D is even, |(m + m')/2]| < 2D/2 and thus |(m+m’)/2] +1 < 2D/2. The last 
case is the one where both m and m’ are odd. Then |m/2]| + |m/’/2] copies are 
added to Hı and Hg, respectively, in the first for loop. In the remaining steps 
of Algorithm 2, two further copies are added to Hı or Hg. In the worst case, 
they both go to one graph, say Hy. Since H fulfills (P5), m+ m’ < 2D. Thus 
[7/2] + |[m’/2| < D—1.Therefore, |m/2| + |m’/2| +2 < D = 2D/2. Thus 
Hı and Ho also fulfill (P4) for D/2. Thus we obtain the next result. 


Lemma 3. Given a multigraph H that fulfills (P1)-(P5) for some D = 2°, 
Algorithm 2 computes two multigraphs Hı and Hz such that both Hı and Ho 
fulfill (P1)-(P4) and for each edge e, my(e) = my, (e) + mH, (e). 


4.3 An Algorithm for Maximum Weight Admissible Multigraphs 


Algorithm 3 repeatedly takes the multigraph H, normalizes it via Algorithm 1, 
decomposes it via Algorithm 2, and proceeds iteratively with the heavier of Hı 
and Hz. Lemmas 2 and 3 immediately prove the following result. 
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Algorithm 3 Maximum weight admissible subgraph 


Input: Multigraph H fulfilling (P1)-(P4) for some even D = 2° 
Output: Maximum weight admissible multigraph S 
for i = 1,...,ġ do 
Normalize H via Algorithm 1. 
Compute multigraphs Hı and Hə from H via Algorithm 2. 
Let w.lo.g. Hı be the heavier of the two computed multigraphs. 
set H = Hı. 
end for 
Set S= H. 


Lemma 4. Given a multigraph H that fulfills (P1)-(P4) for some D = 2° and 
w(H) = w*, Algorithm 3 computes a maximum weight admissible multigraph S. 


4.4 Making D a Power of Two 


If D is not a power of two, then we use the following standard trick and replace 
it by D = 2° where 6 is the smallest natural number such that 2n?D < 2°. Each 
value of the optimal solution z*,, is rounded down to the next multiple of 2~°. 
Let uy be the values obtained. We have 


Dio S r 2m (1) 


Let H be the multigraph that has Puy ° D copies of each edge (u,v). 
Since we round each value down, H fulfills (P1)—(P4). It remains to estimate 
the loss of weight. By (1), 


w(H) = ` wu, v)Byy2° 
(u,v)EK(V) 
>w-  S*—w(u,v)a7F 28 
(u,v)EK(V) 
> woe — n? 


because w(u,v) € {0,1}. If we now run Algorithm 3 on the graph Ê for ô 
iterations, then we end up with an admissible multigraph S of weight 


w(S) > (w*28 — n?) /2 > w* — 1/(2D). 


Therefore, Dw(S) > Dw* — 1/2. Since both quantities Dw(S) and Dw* are 
integers, we have that even Dw(S) > Dw”. 

Therefore, we have a polynomial time solution for computing maximum 
weight admissible multigraphs. 


Theorem 1. On input H , Algorithm 3 computes a maximum weight admissible 
multigraph in polynomial time. 
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Fig. 1. On the lefthand side: A potential 2-cycle on a cycle. On the righthand side: 
The two ways how a potential 2-cycle on a cycle is treated 


5 Coloring Admissible Multigraphs 


We show that for any admissible multigraph G, there is another admissible 
multigraph G” of the same weight that is even 2-path colorable. Given G, G’ can 
be found in polynomial time. Our aim is to exploit the path coloring result by 
Lewenstein and Sviridenko. To do so, we have to deal with 2-cycles on a cycle. 

A 2-cycle € on a cycle locally looks as depicted on the lefthand side of Figure 1. 
If e has weight zero, then we can simply discard e. This does not change the 
weight. The resulting graph is still admissible and c is no longer a 2-cycle on a 
cycle. If e has weight one, then we remove f and add another copy of e. This 
can only increase the weight, since the weight of f is either zero or one. The 
resulting graph is still admissible and c is no longer a 2-cycle on a cycle. (Note 
that the procedure of Lewenstein and Sviridenko can deal with double edges.) 

We now consider all (unordered) pairs of nodes and deal with them as de- 
scribed above. This shows the following result. 


Lemma 5. Given an admissible multigraph G and a weight function w : K(V) 
— {0,1}, there is a 2-path-colorable admissible multigraph G’ with w(G) < 
w(G"). Given G, G’ can be found in polynomial time. 


6 Applications 


Algorithm 4 now computes a maximum path packing or a maximum TSP tour, 
respectively. We first solve the fractional LP, round the optimum solution as in 
Section 4.2, make it 2-path-colorable, and finally take the color class of larger 
weight as a path packing, in the case of DPP, or patch the paths of this class 
together to form a TSP tour, in the case of MaxATSP(Q, 1). 


Theorem 2. Algorithm 4 is a polynomial time 3/4-approximation algorithm for 
DPP and MaxATSP(O, 1). 
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Algorithm 4 DPP, MaxATSP(0, 1) 
Input: Directed Graph G = (V, K(V)) with weight function w : K(V) — {0,1} 
Output: A path packing or TSP tour, respectively 
Solve the fractional LP in Section 3. 
Round the optimum fractional solution H* to an admissible multigraph S via Algo- 
rithm 3. (Replace H* by H, if necessary. ) 
Find a 2-path colorable admissible graph S’ with the same weight as S. 
Color the edges of S’ with two colors such that each color class is a collection of 
node-disjoint paths. 
Return the collection of larger weight. In the case of MaxATSP(0,1), patch these path 
together arbitrarily. 


Proof. By construction, the output of the algorithm is a feasible solution and 
it is computed in polynomial time. It remains to estimate the approximation 
performance. 

If G has a path packing P with £ edges, then there is an admissible multigraph 
of weight 36. To see this, we decompose P into two matchings Mı and Mə by 
placing the edges of each path in P into Mı and Mg in an alternating fashion. 
Let w.lo.g. be Mı the matching with more edges. Then PU M; is an admissible 
multigraph of weight Sf. 

In the same way we see that if there is a TSP tour of weight @, there is 
an admissible multigraph of weight Se. (Strictly speaking, this is only true if 
the number of nodes is even. We can make it even by either adding a dummy 
node that is connected with weight zero edges to all other nodes. This gives a 
multiplicative loss of (1 — 1/n) in the approximation factor. Or we can guess two 
consecutive edges and contract them into one. This increases the running time 
by a quadratic factor, but does not affect the approximation performance.) 

The optimum admissible multigraph is divided into two color classes. There- 
fore, the heavier of the two classes has weight at least Se. O 


Corollary 1. There is a 5/4-approximation algorithm with polynomial running 
time for ATSP(], 2). 


Proof. Vishwanathan [9] showed that any (1 — a)-approximation algorithm for 
MaxATSP(0, 1) yields an (1 + a)-approximation for ATSP(1, 2), too. 0 
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Abstract. We study a variant of the maximum coverage problem which 
we label the maximum coverage problem with group budget constraints 
(MCG). We are given a collection of sets S = {51,S2,...,5m} where 
each set S; is a subset of a given ground set X. In the maximum cov- 
erage problem the goal is to pick k sets from S to maximize the cardi- 
nality of their union. In the MCG problem & is partitioned into groups 
G1, G2,..., Ge. The goal is to pick k sets from S to maximize the car- 
dinality of their union but with the additional restriction that at most 
one set be picked from each group. We motivate the study of MCG by 
pointing out a variety of applications. We show that the greedy algorithm 
gives a 2-approximation algorithm for this problem which is tight in the 
oracle model. We also obtain a constant factor approximation algorithm 
for the cost version of the problem. We then use MCG to obtain the first 
constant factor approximation algorithms for the following problems: (1) 
multiple depot k-traveling repairmen problem with covering constraints 
and (i1) orienteering problem with time windows when the number of 
time windows is a constant. 


1 Introduction 


In this paper we are interested in a variant of the set cover problem and its 
maximization version, the maximum coverage problem. The set cover problem 
is the following: we are given a ground set X of n elements and a set of subsets 
S = {5}, So,..., Sm} such that for 1 < i < m, S; C X. The objective is to find a 
minimum number of sets from S such that their union is X. In the cost version 
of the set cover problem each set S; has a cost c(S;) and we seek a minimum 
cost collection of sets to cover all the elements of X. The maximum coverage 
problem is a close relative of the set cover problem. We are given a ground set 
X and S = {5 4, S2,..., 5m} of subsets of X. In addition we are given an integer 
k and the goal is to pick at most k sets from S such that the size of their union 
is maximized. In the profit version of the maximum coverage problem the items 
in X have profits and the goal is to maximize the profit of items picked. 
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Set cover and maximum coverage problems are fundamental algorithmic 
problems that arise frequently in a variety of settings. Their importance is partly 
due to the fact that many covering problems can be reduced to these problems. 
The greedy algorithm that iteratively picks the set that covers the maximum 
number of uncovered elements is a (Inn + 1) approximation for the set cover 
problem and an 35 approximation’ for the maximum coverage problem [15]. 
Feige [12] showed that these ratios are optimal unless NP is contained in quasi- 
polynomial time. In a number of applications the set system is implicitly defined 
and the number of sets is exponential in the number of elements. However, the 
greedy algorithm can still be applied if a polynomial time oracle that returns a 
set with good properties is available. 

In this paper, motivated by several applications, we study a variant of the 
maximum coverage problem that we call the maximum coverage problem with 
group budget constraints (MCG). We start with a simple motivating example, 
the multiple knapsack problem (MKP), and the analysis of the greedy algorithm 
for that problem in [9]. MKP is a generalization of the classical knapsack problem 
to several knapsacks: we are given n items, where item t has a size s; and a profit 
pi; we are also given m knapsacks potentially of different capacities b1, b2,..., bm. 
The objective is to find a maximum profit subset of items that can be feasibly 
packed into the given set of knapsacks. We first consider the case where all the 
knapsacks have the same capacity 6. For simplicity, in the following discussion, 
we assume that we have an exact algorithm to solve the single knapsack problem. 
We can apply the following greedy algorithm: pick an unused knapsack and use 
the single knapsack algorithm to pack it optimally with a subset of items from the 
remaining items. It is easy to show, via standard set cover style arguments, that 
this algorithm gives an ==; approximation [9]. That the greedy algorithm gives 
this ratio can also be seen via a reduction to the maximum coverage problem as 
follows. Let S be the set of all distinct subsets of the items that can be feasibly 
packed into a knapsack of size 6. The MKP problem can be rephrased as a 
maximum coverage problem on this implicit exponential sized set system and 
we are required to pick m sets. The greedy algorithm that we described for the 
MKP can be seen to be the greedy algorithm for maximum coverage problem 
on the implicit set system above where the oracle is the optimal single knapsack 
algorithm. 

Now we consider instances of MKP in which the knapsack sizes could be 
different. In this case too we can define a greedy algorithm, however since the 
knapsacks are not identical, it is not a priori clear in which order to consider 
them. In [9] it is shown that irrespective of the ordering, the greedy algorithm 
results in a 2-approximation. Once again it is instructive to understand the 
implicit set system. Let S; be set of distinct subsets of items that can be feasibly 
packed into a knapsack of size b; and let S = U;S;. It is clear that if the knapsacks 
are not identical, we no longer have a maximum coverage problem although 
the greedy algorithm still gives a constant factor approximation algorithm. The 








' In this paper approximation ratios for both minimization and maximization problems 
will be greater than or equal to 1. 
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problem we have is the following: we are required to choose at most one set 
from each of the S; and cover as many elements as possible by the union of sets 
picked. 

Motivated by MKP and other problems that we consider later in this pa- 
per, we define a variant of the maximum coverage problem that provides an 
abstraction for covering problems that have groups of sets. 


Maximum Coverage with Group Budgets (MCG): We are given subsets 
S1, S2,..., Sm Of a ground set X. We are also given sets G1,..., Gg, each G; 
being a subset of {S},..., Sm}. We call G; a group. By making copies of sets, if 
necessary, we can assume that the groups G; are disjoint from each other. We 
define two versions of the problem, the cardinality version and the cost version. 

In the cardinality version, we are given an integer k, and an integer bound k; 
for each group G;. A solution is a subset H C {S1,..., Sm} such that |H| < k 
and |HNG;| < k; for 1 <i < £. The objective is to find a solution such that the 
number of elements of X covered by the sets in H is maximized. In fact, we can 
assume without loss of generality that all k; are equal to 1. Otherwise, we can 
make k; copies of each group Gi. 

In the cost version, we associate a cost e(.S;) with each set S}. Further, we 
are given a budget B; for group G;, 1 < i < £, and an overall budget B. A 
solution is a subset H C {S1, S2,..., Sm} such that the total cost of the sets in 
H is at most B. Further for any group G;, the total cost of the sets in H N G; 
can be at most B;. The objective is to find such a subset H to maximize the size 
of the union of sets in H. 

In many applications, m, the number of given subsets of X, is exponential 
in n and the sets are defined implicitly. In such cases we require a polynomial 
time oracle with some desirable properties. We now make this more precise. 
In the cardinality version, we assume there exists an oracle A that takes as 
input, a subset X’ of X, and an index i. A(X’,i) outputs a set S; € G; such 
that |S; X’| is maximized over all sets in G;. We also work with approximate 
oracles. A is an a-approximate oracle if A(X’,2) outputs a set S; € G; such 
that |5; N X’| > + maxpeag, |D N X’|. For the cost version, we shall assume we 
are given an oracle B that takes as input a subset X’ of X and a group index 7. 
B(X’, i) outputs a set S; € G; such that Ba is maximized — we shall assume 
that all sets in G; have cost at most B;. As with the cardinality case we also 
work with approximate oracles. 

We note that the maximum coverage problem is a special case of the car- 
dinality case of MCG and the budgeted maximum coverage problem [17] is a 
special case of the cost version of MCG. 

In this paper we show that a simple greedy algorithm gives a constant fac- 
tor approximation for both the cardinality and the cost versions of MCG. The 
analysis differs from the usual analysis for the maximum coverage problem and 
is based on the analysis for MKP in [9]. For the cardinality version we show that 
greedy gives a 2-approximation and our analysis is tight. For the cost version we 
obtain a 12-approximation. The greedy algorithm works in the oracle model as 
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well and in fact this is the main thrust of the paper and leads to the applications 
that we mention below. 

We note that for the cardinality version, an -=;-approximation is achievable 
if the input is given in an explicit form. This ratio is achieved by rounding an 
LP relaxation either by the pipage rounding technique of Ageev and Sviridenko 
[1] or the probabilistic rounding technique of Srinivasan [18]. 


Set Cover with Group Budgets (SCG): We define the set cover problem 
with group constraints. We only consider the cardinality version in this paper, 
it is easy to extend it to the cost case. We are given a ground set X of n 
items and a set S = {5;,52,...,5m} of subsets of X. The set S is partitioned 
into groups G1,G2,...,Gz. The objective is to find a subset H of S such that 
all elements of X are covered by sets in H and max‘_, |H N G;| is minimized. 
Note that if we have a single group containing all sets then the problem is the 
same as the set cover problem. Elkin and Kortsarz [10] seem to be the first to 
consider this problem for its applications and they call it the multiple set cover 
problem. They present an O(log n) approximation using a randomized rounding 
of a natural linear programming relaxation. Kortsarz [16] asked if there is a 
combinatorial algorithm for this problem. From the 2-approximation bound on 
Greedy we obtain a simple combinatorial (logn + 1) algorithm for SCG. It also 
has the advantage of working in the oracle model. 

We consider two problems for which we design the first constant factor ap- 
proximation algorithms by using reductions to instances of MCG in the oracle 
model. We describe them next. 


Multiple Depot k-Traveling Repairmen Problem: The k-traveling repair- 
men problem was considered by Fakcharoenphol, Harrelson, and Rao [11]. Their 
problem is the following. We are given a finite metric space on a set of nodes 
V induced by an edge weighted undirected graph G, and k not necessarily dis- 
tinct vertices $1, 52,...,$s% from V. A feasible solution to the problem is a set of k 
tours T3,...,Tk, with tour T; starting at s; such that every vertex in V is visited 
by one of the tours. Given a feasible solution, we define the latency of a vertex 
as the time at which it gets visited by one of these tours. The objective is to 
minimize the sum of the latencies of the vertices in V. The problem models the 
case where k-repairmen are available at the k locations (depots) and we need to 
visit all the sites that have repairs. The goal is to minimize the average waiting 
time of the sites. If k = 1, the problem is the same as the the minimum latency 
problem for which a constant factor approximation was first given by Blum et 
al. [5]. The current best approximation ratio for the minimum latency problem 
is 3.59 [8]. In [11] a constant factor approximation algorithm is presented for the 
k-traveling repairmen problem when all the repairmen start at the same vertex 
s, that is s1 = $2 =... = Sk = s. In the same paper [11], the generalization 
of the k-traveling repairmen problem to the case with multiple sources (depots) 
is left as an open problem. We obtain a constant factor approximation for this 
problem. We also obtain constant factor approximation algorithms for several 
generalizations as well. 
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Orienteering (or TSP) with Time Windows: The orienteering problem is 
defined as follows. We are given a metric space on a set of vertices V, a starting 
vertex s and a budget B. A feasible solution is a tour starting at s and having 
length at most B. The objective is to maximize the number of vertices covered by 
this tour. Blum et al. [7] gave the first constant factor approximation algorithm 
for this problem in general graphs, they obtained a ratio of 5 which has recently 
been improved to 3 in [6]. Previously, a 2 + -approximation was known for the 
case when the metric was induced by points in the Euclidean plane [2]. 

In this paper we consider the more general problem where we associate a 
window [ry, dẹ] with each vertex v. A vertex v can be visited only in the time 
interval [r,,d,]. We shall say that rẹ is the release time of v and d, is the deadline 
of v. The objective, again, is to find a path that starts at s and maximizes the 
number of vertices visited, however we can only visit a vertex within its time 
window. This problem models a variety of situations when technicians or robots 
have to visit a number of different locations in a time period. The problem is 
referred to by different names in the literature including prize collecting traveling 
salesman problem with time windows and TSP with deadlines. Tsitsikilis [19] 
showed that this problem is strongly NP-complete even when the metric space is 
a line. Bar-Yehuda et al. [4] gave an O(log n) approximation when the vertices lie 
on a line. Recently Bansal et al. [6] gave an O(log? n) for the general problem. In 
this paper we consider the case when the number of distinct time windows, k, is a 
fixed constant independent of the input. We give a constant factor approximation 
algorithm for this problem using a reduction to MCG and using the algorithm 
in [7, 6] as an oracle. 

The thrust of this paper is the definition of MCG and its applications. In this 
extended abstract we have not attempted to optimize the constants that can be 
achieved for the problems we consider. We defer this to the final version of the 


paper. 


2 Greedy Algorithm for MCG 


In this section we show that simple greedy algorithms give constant factor ap- 
proximation ratios for MCG. First we consider the cardinality version of the 
problem. 


2.1 Cardinality Version 


We can assume without loss of generality that the number of groups £ > k. We 
work in the oracle model and assume that we have an a-approximate oracle. The 
greedy algorithm we consider is a natural generalization of the greedy algorithm 
for the maximum coverage problem. It iteratively picks sets that cover the max- 
imum number of uncovered elements, however it considers sets only from those 
groups that have not already had a set picked from them. The precise algorithm 
is stated below. 


Maximum Coverage Problem with Group Budget Constraints 77 


Algorithm Greedy 
H —@, X'e- xX. 
For 7 = 1,2,...,k do 
For i = 1,..., do 
If a set from G; has not been added to H then A; — A(G;, X") 
Else A; — @ 
EndFor 
r +— argmax;|A;| 
H — HU{A,}, X’ X' — A, 
EndFor 
Output H. 


By renumbering the groups, we can assume that Greedy picks a set from 
group G; in the jth iteration. Let OPT denote some fixed optimal solution and 
let 43 < 42 <... < ik be the indices of the groups that OPT picks sets from. We set 
up a bijection m from {1,2,...,k} to {i1,%2,...,i} as follows. For 1 < hk < k, 
if h € {t1,12,...,%,} then we require that 7(h) = h. We choose m to be any 
bijection that respects this constraint. 

Let C; be the set that Greedy picks from G;, and let O; be the set that 
OPT picks from Gz(;). We let AZ = A; — UJ— An denote the set of new elements 
that Greedy adds in the jth iteration. Let C = U;A; and O = U;O,; denote the 
number of elements that Greedy and OPT pick. 


Lemma 1. For 1 < j < k, |Ai| > |0; — C|. 


Proof. If O; —C = @ there is nothing to prove. When Greedy picked A;, the set 
O; was available to be picked. Greedy did not pick O; because |Aj | was at least 


+10; ~— U Anl. Since U An C C, the lemma follows. 0 


Theorem 1. Greedy is an (a+ 1)-approzimation algorithm for the cardinality 
MCG with an a-approximate oracle. 


Proof. From Lemma 1, we have that 


J 


el = E14; 2 =O; = C| > 20/4; O41 = Ie) = = (101 = 1C). 


Hence |C| > iO]. 0 


4 
a+l 
Corollary 1. [fk = £, Greedy is an (a+ 1)-approzimation algorithm even if it 
is forced to pick sets from an adversarially chosen ordering of the groups. 


Proof. If k = £, the permutation m is the identity permutation. In this case 
Lemma 1 holds again. 0 


Easy examples show that our analysis of the Greedy algorithm is tight. In 
fact, in the oracle model, the ratio of 2 cannot be improved. When the set system 
is available as part of the input, the problem is hard to approximate to within 
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a factor of z$% via a reduction from the maximum coverage problem. As we 
mentioned earlier, a matching ratio can be obtained via linear programming 


[1,18]. 


A logn+1 Approximation for SCG: We observe that the 2-approximation 
bound for MCG can be used to obtain a logn + 1 approximation for SCG as 
follows. We simply guess the optimal value A*, that is there is an optimal cover 
H* such that max; |H*NG;| < A*. We then create an instance of MCG by having 
a budget of A* on each group G;. Greedy covers at least 1/2 the elements in X. 
Iterating Greedy logn + 1 times results in a solution that covers all elements. In 
each iteration the number of sets added from any given group is upper bounded 
by A*. Hence, when all elements are covered, the number of sets added from any 
group is at most (logn + 1)A*. 


2.2 Cost Version 


We now consider the cost version of MCG. We give a greedy algorithm for this 
problem which is similar in spirit to the one for the cardinality case but differs 
in some technical details. The algorithm that we describe below may violate the 
cost bounds for the groups or the overall cost bound B. We will later show how to 
modify the output to respect these bounds. We work in the oracle model again. 
Recall that the oracle A, given a set of elements X’ and an index 7 returns a set 
S € G; that approximately minimizes the ratio maxpec, BH. The algorithm 
is described in more detail below. We assume without loss of generality that 


B S ya Bj. 


Algorithm CostGreedy 
H—-@, X'-X. 
Repeat 
For i= 1,2,...,/ do 
If e(H N G;) < Bi then A; — A(X’, Gi). 
Else A; — Í. 
EndFor 
r — argmax, Jh 
H — HU{A,}, X — X'— Ar. 
Until (c(H) > B). 
Output H. 


Note that H need not obey the budget requirements. Define H; = H N G;. 
For a set S chosen by the algorithm, define X(S) as the extra set of elements in 
X that are covered at the time S is added to H. Define X(H;) = Usen; X (S). 
Similarly, X(H) is the set of elements covered by the algorithm. Let OPT be some 
fixed optimal solution to the given problem instance. Let O be the set of sets 
chosen by OPT. Define Y; = O N G;. We call an index i good if e(H N G:) < B;, 
that is the algorithm did not exceed the budget for G;. Otherwise we call i bad. 
We omit proofs of the following lemmas in this version. 
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Lemma 2. Jf i is bad, then |X(H;)| > 4D aey, 





A- X(H)|. 
Corollary 2. |X(H)| > 34l Ui: bad Uaer:Al: 


Lemma 3. |X(H)| > 4 Ui ; good Uae. Al. 





From Corollary 2 and Lemma 3, it follows that X(H) > Mari OPT. But H 
does not respect all the budget constraints. We partition H into three subsets 
H, Ho, Hz. H3 is the last set picked by our algorithm. Hz contains those sets S$ 
which when added to H caused the budget of some group G; to be violated — 
however we do not include the set in H3. Hı contains all the remaining sets in 
H. It is easy to see that Hı, Hə and H3 do not violate the budget constraints. 
Further, one of these three sets must be covering at least 1/3 the number of 


elements covered by H. Thus we get the following theorem. 


Theorem 2. The algorithm CostGreedy is a 6(a + 1)-approzimation algo- 
rithm for the cost version of MCG. 


3 Applications of MCG 


3.1 The k-Traveling Repairmen Problem 


Recall from Section 1 that in the k-traveling repairmen problem we are given a 
metric space on a set of nodes V induced by an edge weighted undirected graph 
G. We are given a set of k source vertices in V, call them s;,89,..., 8%. A feasible 
solution to the problem is a set of k tours, one tour starting at each source s4, 
such that every vertex in V is visited by one of the tours. Given a feasible 
solution, we define the latency of a vertex as the time at which it gets visited by 
one of these tours. The objective is to minimize the sum of the latencies of the 
vertices in V. 

We give the first constant factor approximation algorithm for the multiple 
depot k-traveling repairmen problem. We define a related problem, which we 
call the budgeted cover problem, as follows. The input to the problem is a subset 
V’ of V and a positive integer B. A solution is a set of k tours, one tour starting 
at each source s;, such that no tour has length more than B. The objective is 
to maximize the number of vertices of V’ covered by these tours. We can view 
this problem in the framework of MCG as follows. The ground set is the set of 
vertices in V’. For each source s;, we have a collection of sets Sj,...,53.: each 
set corresponds to a distinct tour of length at most B beginning at s;. There are 
k groups, one group for each source vertex s;. The group G; corresponding to 
ole tol eee S$}: Clearly, the cardinality version of MCG for this set system is 
the same as the budgeted cover problem for the original graph. From Section 2 
we will get a constant factor approximation algorithm for the budgeted cover 
problem provided we have the following oracle A. 

The oracle A should be able to solve the budget-MST problem. In this prob- 
lem, we are given a graph G = (V, E), a source s € V and a budget B. A solution 
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is a tour starting at s and having cost at most B. The objective is to maximize 
the number of vertices covered by the tour. Unfortunately the budget-MST prob- 
lem is NP-hard. However, by using the algorithm for the i-MST problem [13, 
3], we can obtain a polynomial time algorithm, A, which covers OPT vertices 
and costs at most ĝ + B. Here 8 > 1 is the approximation ratio for the i-MST 
problem. Hence, if we are willing to violate the budget constraint by a factor of 
Ê we can cover as many vertices as the optimal solution. 

It follows from Theorem 1 that we can get a polynomial time algorithm for 
the budgeted cover problem which covers at least half as many vertices as the 
optimum, and constructs k tours, each tour of length at most 6B. We call this 
algorithm C. 

We can now describe our algorithm for the traveling k-traveling repairmen 
problem. Our algorithm works in phases. We assume without loss of generality 
that all distances are at least 1. Let V; be the set of uncovered vertices at the 
beginning of phase j (so Vo = V). In phase j, we cover as many vertices as 
possible so that the budget of each tour is about 2%. More precisely, we do the 
following 


Algorithm Visit(j) : 

For p = 1,2 do 
Run C on the budget cover problem instance with inputs Vj and 27. 
Remove from V; the covered vertices. 


This describes phase j. We invoke the subroutine Visit(j) with increasing 
values of 7 until all vertices are covered. Given a source s;, we have constructed 
several tours starting from s;. We just stitch them together starting in the order 
these tours were found by the algorithm. Clearly, our algorithm produces a 
feasible solution. It remains to prove that it is a constant factor approximation 
algorithm. 

We begin with some notation. Fix an optimal solution OPT. Let O; denote 
the set of nodes in OPT’s solution which have latency at most 27. Let C} be the 
set of nodes visited by our algorithm by the end of phase 7. 


Lemma 4. Visit(j) covers at least 30; — Cj—-1| vertices. 


Proof. Let R; denote O; —C;_1. Let A; be the set of nodes covered by Visit(7) 
when p = 1. Theorem 1 implies that |A;| > $|R,;|. One more application of this 
theorem when p = 2 gives the desired result. o 


The rest of the proof goes along the same lines as in [11]. Let n; be the set 
of nodes in OPT whose latency is more than 27. Let n; be the set of nodes in our 
tour which do not get visited by the end of phase 7. 


/ Lal Ss 


Proof. From Lemma 4 it is easy to see that nj < nj_, —3/4|0; — Vj~1|. Clearly, 


|O;| =n —n,; and |Vj-1| =n —n_,. Combining these proves the lemma. O 
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The total latency of the tours obtained by our algorithm is upper bounded 
by pay. 4821n} and that produced by the tours in the optimal solution is lower 
bounded by $; 27~*n;. From Lemma 5 we obtain that 


> Fn, < ae +i n; 
j j j 


which implies that 


Sn <3.) i 
j j 


This proves that our algorithm yields a 126 approximation. We can improve 
the ratio by using ideas from [14,11,8]; we defer the details to the final version. 
We can also obtain constant factor approximations for each of the following 
generalizations: (1) each vertex v can be serviced only by a given subset 5S, of 
repairmen, (11) each vertex v has a service time p, that the repairmen needs to 
spend at the vertex, and (111) each vertex v has a weight w, and the objective is 
to minimize the sum of the weighted latencies. 


3.2 The Orienteering Problem with Time Windows 


We now consider the orienteering problem with time windows. We assume that 
the number of distinct time windows is some fixed constant k. We use, as a 
subroutine, the algorithm of Bansal et al. [6] which provides a 3-approximation 
for the case when there is a single time window [0, D] for all vertices and the tour 
is required to start at a vertex s and end at a vertex t. In the rest of the section 
we use ĝ to denote the approximation ratio for the single deadline case. All our 
ratios will be expressed as functions of 8. Let A be the maximum distance in the 
metric space. We begin by describing approximation algorithms for two special 
cases: (1) when all release times are zero, and (2) when all deadlines are the 
same. 


Release Times Are Zero: We consider the special case when r, = 0 for all 
nodes v. Let dı < dy < ... < dk be the k distinct deadlines. Let V; denote the 
set of vertices whose deadline is d;. Let P* be the tour constructed by some 
optimal solution. Define vg as the source vertex s. For 1 < i < k, let v7 as the 
last vertex in the tour P* which is visited by the deadline d;. It is possible that 
vš = v% for two distinct indices 7 and i’. Suppose uy is visited at time tř, then 
it follows that that t} < d;. 
Our algorithm first guesses the vertices vj, v3,...,v,; and the time instances 
+,t4,...,t%. Note that t* < nA. Hence the total number of guesses is O(n?* A*). 
Since Aneed not be polynomially bounded, the number of guesses is not poly- 
bounded. We omit details on how to use a polynomial number of guesses. Now, 
wedefine k groups G1,...,G asfollows. G; is the set of all paths on the vertex 
set V; U Vi+1 U- +- U Vk which originate at vž_ and end at v7 with the additional 
constraint that the length of the path is at most tł — ¢7_,. 
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Lemma 6. Consider the instance of the MCG with groups defined as above 
where we need to pick exactly one set from each group. A *y-approzimation. al- 
gorithm for this problem instance implies the same for the corresponding orien- 
teering problem. 


Proof. Suppose we are given a solution to the MCG which picks paths P;,..., Pk 
from the corresponding groups. If we stitch these tours sequentially, it is easy 
to see that we get a path which satisfies the deadlines of the vertices visited by 
the individual paths. Therefore the number of vertices covered by this tour is 
|P; U Po U+- -U Ppl. Further, if we consider the tour P*, we can get a solution to 
the MCG which covers |P*| vertices. This proves the lemma. O 


Thus, it is enough to approximate the MCG induced by the guess of vj,..., v% 
and tj,...,¢z. The oracle needed for this instance is an algorithm for the ori- 
enteering problem where the time windows for all the nodes are of the form 
[0,D], hence we can use the algorithm of Blum et al. [7] or Bansal et al. [6]. 
From Corollary 1 we obtain a a (8 + 1)-approximation algorithm for the case 
of k deadlines. The running time of the algorithm is O(n?* A*T) where T is the 
running time of the approximation algorithm for the single deadline case. 


Single Deadline: We now consider the special case when all deadlines are the 
same, say D but the release dates can be different. Consider any feasible tour P 
which starts at s and ends at a vertex u. Let the length of P be ¢(P). Suppose we 
reverse the tour, i.e., we view the tour as a path P” starting at u and ending at s. 
If P visits a vertex v at time t, then P” visits v at time @(P)—t.Sor, < t < £(P) 
implies that 0 < (P) —t < (P) —r,. Thus, we can view this tour as one in 
which the release time of a vertex is O and the deadline is (P) —r,. Therefore, if 
we could guess the length of the optimal path P* and the last vertex u* in this 
path, then we could just use the algorithm mentioned in the previous section. 
Thus, we can get a (8 + 1)-approximation algorithm for this problem as well. 
The running time of the algorithm increases by a factor of A from the algorithm 
for single release date. 


k Time Windows: Now we address the case where there are both release times 
and deadlines. Let rı < rg < ... < rg be the k distinct release time and let 
dı < dg <... < dk be the k distinct deadlines. Let P* be the tour constructed 
by the optimal solution. As before let vý be the last vertex in P* to be visited 
before d; and let ty be the time at which vj is visited. Recall that V; is the 
set of vertices with deadline d;. We define group G; to be the set of tours that 
start at v7_, at ¢*_, and end at vř by tš. The vertices that a tour in G; can 
visit are constrained to be in V; UV;4; U... U Vk. Lemma 6 trivially generalizes 
to this setting to yield a y approximation provided we have an oracle for the 
MCG instance. Consider a group G;. The vertices all have a deadline at least as 
large as tř, hence we have a single deadline. The vertices might have different 
release times, however there are at most & distinct release times. Hence the oracle 
needed for G; can be obtained from the algorithm described above for this case 
that has an approximation ratio of 6+ 1. Thus, once again applying Theorem 1 
we can obtain a (@+2)-approximation algorithm for the case of k time windows. 
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Theorem 3. Given a B-approximation algorithm for the orienteering problem 
with a single time window for all vertices, there is a (B + 2)-approximation al- 
gorithm for the orienteering problem with at most k distinct time windows that 
runs in time polynomial in (nA)*. 
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Abstract. In the Minimum Common String Partition problem (MCSP) 
we are given two strings on input, and we wish to partition them into the 
same collection of substrings, minimimizing the number of the substrings 
in the partition. Even a special case, denoted 2-MCSP, where each let- 
ter occurs at most twice in each input string, is NP-hard. We study a 
greedy algorithm for MCSP that at each step extracts a longest common 
substring from the given strings. We show that the approximation ratio 
of this algorithm is between 2(n°**) and O(n®®?). In case of 2-MCSP, 
we show that the approximation ratio is equal to 3. For 4-MCSP, we give 
a lower bound of (log n). 


1 Introduction 


By a partition of a string A we mean a sequence P = (P;, Po,..., Pm) of strings 
whose concatenation is equal to A, that is P,P)... Pm = A. The strings FP; are 
called the blocks of P. If P is a partition of A and Q is a partition of B, then 
the pair r = (P, Q} is called a common partition of A, B, if Q is a permutation 
of P. For example, m = ((ab, becad, cab), (bccad, cab, ab)) is a common partition 
of strings A = abbccadcab and B = bccadcabab. 

The minimum common string partition problem(MCSP)is defined as follows: 
given two strings A, B, finda common partition of A, B with the minimal number 
of blocks, or report that no common partition exists. By k-MCSP we denote the 
version of MCSP where each letter occurs at most k times in each input string. 

The necessary and sufficient condition for A, B to have a common partition 
is that each letter has the same number of occurrences in A and B. Strings with 
this property are called related. Verifying whether two strings are related can be 
done easily in linear time, and for the rest of the paper we assume, without loss 
of generality, that the input strings are related. 

In this article, we deal with approximations of MC SPP. In particular, we study 
the greedy algorithm for MCSP that constructs a common partition by iteratively 
extracting the longest common substring of the input strings. More precisely, the 
algorithm can be described in pseudo-code as follows: 


K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 84—95, 2004. 
© Springer-Verlag Berlin Heidelberg 2004 
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Algorithm GREEDY 
Let A and B be two related input strings 
while there are symbols in A or B outside marked blocks do 
S e longest common substring of A, B that does not 
overlap previously marked blocks 
mark one occurrence of S$ in each of A and B as blocks 
(P,Q) sequence of consecutive blocks in A and B, respectively 


For example, if A = cdabcdabceab, B = abceabcdabcd, then GREEDY first 
marks substring abcdabc, then ab, and then three single-letter substrings c, d, e, so 
the resulting partition is ((c, d, abedabe, e, ab), (ab, c, e, abedabc, d)), while the op- 
timal partition is ((cdabed, abceab), (abceab, cdabcd)). As illustrated by the above 
example, the common partition computed by GREEDY is not necessarily optimal. 
The question we study is what is the approximation ratio of GREEDY on MCSP 
and its variants. We prove the following results (by n we denote the length of 
the input strings): 


Theorem 1.1. (a) The approximation ratio of GREEDY for MCSP is between 
2(n°-43) and O(n?-69), 

(b) For 4-MCSP, the approximation ratio of GREEDY is at least Q(logn). 

(c) For 2-MCSP, the approximation ratio of GREEDY is equal to 3. 


Our results can be extended to the variation of the minimum common parti- 
tion problem, where letters have signs associated with them (cf. [1]). A substring 
P may now appear in A either as P or as its reversal P£, where in the rever- 
sal the signs of all letters are reversed as well. As in MCSP, we want to find a 
minimum common partition of A and B into such substrings. 


Related Work. The minimum common string partition problem was introduced 
by Chen et al. [1]. They pointed out that MCSP is closely related to the well- 
known problem of sorting by reversals and they use MCSP for comparison of 
two DNA sequences. In this application, the letters in the alphabet represent 
different genes in the DNA sequences, and the cardinality of the minimum com- 
mon partition measures the similarity of these sequences. The restricted case of 
k-MCSP is of particular interest here. Goldstein et al. [3] proved that 2-MCSP 
is NP-hard. 

The size of the minimum partition of A and B can be thought of as a distance 
between A and B. The classical edit-distance of two strings is defined as the 
smallest number of insertions, deletions, and substitutions required to change one 
string into another [5]. Kruskal and Sankoff [4], and Tichy [8] started to consider 
block operations in string comparison, in addition to the character operations. 
Lopresti and Tomkins [6] investigated several different distance meassures; one 
of them is identical to the MCSP measure. 

Shapira and Storer [7] study the problem of edit distance with moves in 
which the allowed string operations are the following: insert a character, delete 
a character, move a substring. They observe that if the input strings A, B are 
related, then the minimum number of the above listed operations needed to 
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convert A into B is within a constant factor of the minimum number of only 
substring movements needed to convert A into B; the later is within a constant 
factor of the minimum common partition size. Shapira and Storer also considered 
a greedy algorithm nearly identical to ours and claimed an O(log n) upper bound 
on its approximation ratio; as it turns out, however, the analysis is flawed. 

Cormode and Muthukrishnan [2] describe an O(log n log* n)-approximation 
algorithm for the problem of edit distance with moves. As explained above, 
this result yields an O(log n log* n)-approximation for MCSP. Better bounds for 
MCSP are known for some special cases. A 1.5-approximation algorithm for 
2-MCSP was given by Chen et al. [1]; a 1.1037-approximation algorithm for 2- 
MCSP and a 4-approximation algorithm for3-MCSP were given by Goldstein et 
al. [3]. All these algorithms are considerably more complicated than GREEDY. 
Due to its simplicity and ease of implementation, GREEDY is a likely choice for 
solving MCSP in many practical situations, and thus its analysis is of its own 
independent interest. 


2 Preliminaries 


By A = aja2...a, and B = b,b2...b, we denote the two arbitrary, but fixed, 
input strings of GREEDY. W.1.0.g., we assume that A and B are related. If 7 is 
a common partition of A, B, then we use notation #blocks(7) for the number of 
blocks in r, that we refer to as the size of m. The size of a minimum partition 
of A, B is denoted by dist(A, B). 

We typically deal with occurrences of letters in strings, rather than with 
letters themselves. By a “substring” we mean (unless stated otherwise) a spe- 
cific occurrence of one string in another. Thus we identify a substring S = 
ApAp41.-.Ap4s Of A with the set of indices {p,p+1,...,p+s} and we write 
S = {p,p+1,...,p+s}, where [S| = s+1 is the length of S. Of course, the 
same convention applies to substrings of B. If S is a common substring of A, B, 
we use notations S4 and SË to distinguish between the occurrences of S in A 
and B. 


Partitions as Functions. Suppose that we are given a bijection € : [n] — [n] 
(where [n] = {1,2,...,}) that preserves letters of A and B, that is, be) = ai 
for all i € [n]. A pair of consecutive positions i,7-+ 1 € [n] is called a break of 
E if éli +1) £ E(t) +1. Let #breaks(€) denote the number of breaks in £. For a 
common substring S of A, B, say S = QpGp41..-Ap4s = bgbg41... 0945, We Say 
that £ respects S if it maps consecutive letters of S4 onto consecutive letters in 
SB, thatis, (i) =i +q — pfor i € SA. 

A letter-preserving bijection € induces a common partition (also denoted 
€, for simplicity) whose blocks are the maximum length substrings of A that 
do not contain breaks of €. The partition obtained in this way does not have 
any “unnecessary” blocks, i.e., #blocks(£) = #breaks(€) + 1. And vice versa, if 
ma = (P, Q) is a common partition of A, B, we can think of 7 as a letter-preserving 
bijection m : [n] — [n] that respects each block of the partition. Obviously, we 
then have #blocks(m) > #breaks(7) +1. We use this relationship throughout the 
paper, identifying common partitions with their corresponding functions. 
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Reference Partitions. Let 7 be a minimum common partition of A and B. 
(This partition may not be unique, but for all A, B, we choose one minimum 
common partition in some arbitrary way.) In the first step, GREEDY is guaran- 
teed to find a substring Sı of length at least the maximum length of a block in 
qt. For the analysis of GREEDY, we would like to have a similar estimate for all 
later steps, too. However, already in the second step there is no guarantee that 
GREEDY finds a substring as long as the second longest block in 7, since this 
block might overlap Sı and it may be now partially marked (in A or B). To get 
a lower estimate on |S;|, for t > 1, the idea is introduce a corresponding refer- 
ence common partition of A, B that respects all the blocks S;,...,5:~-1 selected 
by GREEDY in steps 1 to t — 1. This partition may gradually “deteriorate” (in 
comparison to the minimum partition of A and B), that is, it may include more 
blocks and its blocks may get shorter. Furthermore, it may not be a minimum 
common partition of the unmarked segments. It is only used to estimate the 
“damage” caused by GREEDY when it makes wrong choices (that is, when it 
marks strings which are not in the optimum partition). 

Denote by g the number of steps of GREEDY on A, B. For t = 0,1,...,g, 
the reference common partition p; is defined inductively as follows. Initially, 
po = T. Consider any t = 1,...,g. Suppose that SA = {p,p+1,...,p +s} and 
SP = {q,q+1,...,q+}. Define function ô : SA — SP such that 6(i) = it+q—p 
forj € SA. Then p; is defined by 


. fêl) fori € SA 
pr(t) = eee (i) for i € [n] — SA (1) 


where (i) = min {1 > 0: pe_1(6~1pe_1)'(é) Z SP}. We now show that each p; 
is well-defined and bound the increase of the number of breaks from p¢— 1 to pr: 


Lemma 2.1. For each t = 1,...,g, (a) pe is a common partition of A, B, (b) 
pe respects S1,..., St, (C) #breaks( pz) < #breaks(pt—1) + 4. 


Proof. The proof of the lemma is by induction. For t = 0, (a) and (b) are trivially 
true. Suppose that ¢ > 0 and that the lemma holds for t—1. To simplify notation 
let S = Si, p = pe and p’ = py. 

Consider a bipartite graph G C [n] x [n], with edges (2, p(z)), for i € [n], and 
(i,6(4)), for i € S4. These two types of edges are called p-edges and 5-edges, 
respectively. 

Let S4 = |n] ~ S4 and 5P = [n] — SP. In this proof, to avoid introducing 
additional notation, we think of S4 and S4 as the sets of nodes on the “left- 
hand” side of G and SP and Ë as the nodes on the “right-hand” side. Then, 
any node in $4 or 5% is incident to one p-edge, and each node in $4 or 9 is 
incident to one p-edge and one é-edge. Thus, G is a collection of vertex disjoint 
paths and cycles whose edges alternate between p-edges and é-edges. We call 
them G-paths and G-cycles. All G-cycles have even length and contain only 
nodes from $4 and SP. All maximal G-paths have odd lengths, start in 54, 
end in SP, and their interior vertices are in S or S?. The G-path starting at i 
has the form i, p(i), 6~'p(i), p57! (i), ... , p(571p)* (i). Thus, for i € $4, p’(i) 
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Fig. 1. An example illustrating the construction of p’. The left part shows p and some 
G-paths. The right part shows p’. The strings in the partitions are numbered, and the 
common substring S; = abecababd is shaded. 


is simply the other endpoint of the G-path that starts at 7. This implies that p 
is l-1 and letter-preserving, so it is indeed a common partition. Condition (b) 
follows immediately from the inductive assumption and the definition of p’. It 
remains to prove (c). 


Lemma 2.2. Suppose that 1,i+1 is a break of p’. Then there is l such that 
l < min {@(7), 2(¢ + 1)} and one of the following conditions holds: 

(BO) Exactly one of i1,i+1 is in SA. 

(B1) 1,4 +1¢€ S4 and (5~'p)'(t), (6-1p)'(i +1) is a break of p. 

(B2) ii+1 € S4 and exactly one of p(d~*p)'(i), p(6~1p)'(i +1) belongs to SP. 


We refer to breaks (BO), (B1), (B2), respectively, as breaks induced by the 
endpoints of $4, breaks induced by the breaks inside $4 (only if i,i+1 is a new 
break), and breaks induced by the endpoints of $8. 


Proof. If exactly one of i, i+ 1 is in S4, the case (BO) holds. Since i, i+ 1 is 
never a break in p’ if both į and i + 1 are in $4, we assume that ii +1 € S4 
for the rest of the proof. 

Consider the largest integer 1 < min {&(i), €(¢+1)} for which (871p)! (i), 
(6-1 p)'(i +1) are consecutive in S4, that is (8671p) (i +1) = (6—-1p)'(i) + 1. 
Let j = (`!) (i). We have two sub-cases. If p(j +1) Æ p(j) +1, then 7,7 +1 is 
a break of p, so condition (B1) is satisfied. If p(j +1) = p(j) +1, then at least 
one of plj), p(j +1) must be in SP, for otherwise i, i +1 would not be a break 
of p’. But we also cannot have both p(j), p(j +1) € S®, since then (871p)! +} (i), 
(6~1p)'+1(4+1) would be consecutive in $4, violating the choice of l. Therefore 
case (B2) holds. a 


We now complete the proof of part (c) of Lemma 2.1. There are no breaks 
of p' inside S4, and we have at most two breaks of type (BO) corresponding 
to the endpoints of $4. By the disjointness of G-paths and cycles, there are at 
most two breaks of p' of type (B2), each corresponding to one endpoint of $P. 
Similarly, each break of p (inside or outside S4) induces at most one break of p’ 
of type (B1). This implies (c), and the proof of the lemma is now complete. O 


Note that we did not use the fact that S has maximum length. So our con- 
struction of p+ can be used to convert any common partition 7 into another 
partition 7’ that respects a given common substring S, and has at most four 
more breaks than 7. 

Lemma 2.1 implies that in every step t, every block in p is either completely 
marked or unmarked. 
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3 Upper Bound for MCSP 


In this section we show that GREEDY’s approximation ratio is O(n®-®°). The 
proof uses reference common partitions introduced in Section 2 to keep track of 
the length of the common substrings selected by GREEDY. 

For p > q > 1, wedefine H(p,q) to be the smallest number h with the 
following property: for any input strings A, B, if at some step t of GREEDY there 
are at most p unmarked symbols in A and at most q unmarked blocks in the 
current reference partition p+, then GREEDY makes at most h more steps until it 
stops (so its final common partition has at most t+h blocks.) For convenience, we 
allow non-integral p and q in the definition. Note that H (p,q) is non-decreasing 
in both variables. 

Before proving the O(n upper bound, note that Lemma 2.1 immediately 
gives a recurrence H(p,q) < H(p(1 — 1/q),q¢+3) + 1 (whenever both values of 
H are defined), as in one step of GREEDY, the longest common substring has at 
least i letters (which will be marked in the next partition), and the number of 
unmarked blocks in the reference partition increases by at most 3. With some 
calculation, this gives a slightly weaker but simpler bound of O(n? 75). 

The idea of the proof of the improved bound is to consider, instead of one 
step of GREEDY, a number of steps proportional to the number of blocks in 
the original optimal partition, and show that during these steps GREEDY marks 
a constant fraction of the input string. This yields an improved recurrence for 


H(p,q). 


Lemma 3.1. For all p,q satisfying p > 9q/5+3, we have 
H (p,q) < H(5p/6, (3q + 5)/2) + (q + 5)/6. 


Proof. Consider a computation of GREEDY on A, B, where, at some time t (1.e., 
after t blocks have already been marked), there are punmarked symbols in A, and 
q unmarked reference blocks of p+. We denote these blocks by R1, R2,..., Rg, in 
the order of non-increasing length, that is |Rz| > |R,41|,for z = 1,...,q— 1. We 
analyze the computation of GREEDY starting at time t. Let g be the number of 
additional steps that GREEDY makes. Our goal is to show that g < H(5p/6, (8q¢+ 
5)/2) + (q + 5)/6. (Since the bound is monotone in p and q, we do not need to 
consider the case of fewer than g unmarked blocks or fewer than p unmarked 
symbols.) If g < (q + 5)/6, this trivially holds, so in the rest of the proof we 
assume that g > (q + 5)/6. 

Let T; = St; be the common substring selected by GREEDY in step t + i. 
We say that GREEDY hits Rz in step t+ iif T; overlaps Rz, either in A or in B, 
that is, if either TAN RÅ 40 or TË N RË Æ Í. 

Claim A. For all j = 1,...,g, the total length of the blocks Rı,...,Rą that are 
hit by GREEDY in A in steps ¢+1,...,¢+ 7 is at most 654 _, IT;l. 

We estimate the total length of the blocks R, that are hit at step t +i in A 
but have not been hit in steps t+ 1,...,t+:i— 1. The total length of the blocks 
that are contained in SA and SP is at most 2|T;|. There are up to four blocks 
that are hit partially, but by the greedy choice of T;, each has length at most 
\T;|, and the claim follows. o 


ata | 
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Claim B. 6 YLE] IT] > p 
Let £ be the minimum integer such that 6 D T = p. Since f alh =D, 


£ is well defined and £ < g. For j = 1,...,£, define A; as the maximal index 
for which YL} |Re| < 65227) |T;l. Since 6 ZEZ IT; < p = q {Re}, all A; 


are well defined, and Ag < q. We also note that Ay = 0. For each j = 1,...,2, 
Claim A implies that one of the blocks R,,...,A),;41 is not hit by any of the 
blocks Tı, ...,Tj-1 and thus, by the definition of GREEDY and the ordering of 
the blocks Rz, {Z| > |Ay,+41|. Considering again the ordering of the blocks Rz, 
we have 6|T;| > |Ry;41] +... + |Ra;+6|. We conclude that Aj;41 > A; + 6, for 
j =1,...,@—1. This, in turn, implies that q > A2+1 > 6€—5.So £ < (¢+5)/6, 
and Claim B follows, by the choice of £ and its integrality. oO 

By Claim B, after exactly |(q + 5)/6] steps, GREEDY marks at least p/6 
letters, so the number of remaining unmarked letters is at most p' = 5p/6. By 
Lemma 2.1, the number of unmarked blocks increases by at most 3 in each step 
(since one new block is marked), so the number of unmarked blocks induced by 
GREEDY in these | (q + 5)/6] steps is at most 3| (q + 5)/6] < (q¢+5)/2. Thus the 
total number of unmarked blocks after these steps is at most qg’ = q + (q +5)/2 = 
(3q + 5)/2. The condition in the lemma guarantees that H(p’, gq’) is defined, so, 
by induction, the total number of steps is at most H(p’, gq’) + (q + 5)/6. O 


Finally, we prove the upper bound in Theorem 1.1 (a). We prove by induction 
on p that for p > q and a sufficiently large constant C, H(p,q) < Cp’(q+ 
5)!-7 — q. We choose C so that for all q < p < 9q/5 + 3, the right-hand side 
is at least p and thus the inequality is valid. For p > 9q/5 + 3, by Lemma 3.1, 
the inductive assumption, and the choice of y, we have H (p,q) < H(2p, 3945) + 
15 < O(Šp) (3 (qa +5)) 7 — $- + SE = Cp(q +5)!7? — dg. Let A, B be 
input strings of with length n and dist( A, B) = m. Then the number of blocks 
in GREEDY’s partition is at most H (n, m) = O(n7)m, and the theorem follows. 


4 Lower Bound for MCSP 


We show that the approximation ratio of GREEDY is (n!/!9825) = N(n®4), 
We first construct strings C;, Di, Ei, F; as follows. Initially, Co = a and Do = b. 
Suppose we already have C; and D;, and let X; be the set of letters used in C4, 
D;. Create a new alphabet X; that for each a € X has a new letter, say, a’. 
We first create strings Æ; and F; by replacing all letters a € X in C; and D;, 
respectively, by their corresponding letters a’ € X. Then, let 


Cit = Ci Di Ei DiC, and Dis = D; E;F; E; D;. 


For each i, we consider the instance of strings A; = CiD; and B; = D;C;. 
For example, Eo = c, Fo = d, Ao = ab, Bo = ba, Cı = abcba, Dı = bedcb, 
A, = abcbabcdcb, and Bı = bedcbabcba, etc. 

Let n = 2-5*. We have |4;| = |B;| = n and dist(A;, Bi) < 2. We claim that 
GREEDY’s common partition of A; and B; has 2*+? —2 = Q(n1/!°825) substrings. 
We assume here that GREEDY does not specify how the ties are broken, that 1s, 
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whenever a longest substring can be chosen in two or more different ways, we 
can decide which choice GREEDY makes. 

The proof is by induction. For 1 = 0, GREEDY produces two substrings, as 
claimed. For i > 0, 


Aii = Ci D; E; DC DE Fi E: Di, Bi = D; Ei Fi E; Di Ci D; E; Di C. 


There are three common substrings of length 5i+1; C,D,E,;DiCi, D;E;F;E;D;:, 
and £;D;C;D,;E;, and no longer common substrings exist. To justify this, we 
use the fact that the alphabet of C;, Di is disjoint from the alphabet of &,, 
F;. Suppose that S is a common substring of length at least 5‘+!. To have this 
length, S must contain either the first or the second E; from Ai;1. We now have 
some cases depending on which £; is contained in S, and where it is mapped 
into B;4; via the occurrence of S in Bi+1. If S contains the first E;, then, by the 
assumption about the alphabets, this &; must be mapped into either E;F; E; or 
into the last FE; in Bi+1. Ifit is mappedinto £;F;£;, then S must be £;D;C;D;£;. 
If it is mapped into the last E; in Bj, 1, then S must be C;D;E;D;C;. In the last 
case, S contains the second E£;. By the same considerations as in the first case, 
it is easy to show that then S must be either DE; F; ED; or E;D;C,D,E£,. 

Breaking the tie, assume that GREEDY marks substring £;DiC;D;E;. The 
modified strings are: 


Ci Di Ei D; Ci D; E; Fi Ei Di, D; E; Fi E; D; Ci D; Ei D; Ci, 


where the overline indicates the marked substring. In the first string the un- 
marked segments are A;, A; D;, and in the second string the unmarked segments 
are B; and D;B;,where A; = F; E; and B; = E;F; are identical as A;, B; respec- 
tively, but with the letters renamed. The argument in the previous paragraph 
and the disjointness of the alphabets implies that the maximum length of a non- 
marked common substring is now 5°. We break the tie again, and make GREEDY 
match the two D,’s in F;E;D; and D;E;F;, leaving us with non-marked pairs 
of substrings {A;, Bi} and {A}, B!}. By induction, GREEDY produces 2*+? — 2 
substrings from A;, B;, and the same number from Aj and B;. So we get the 
total of 2(2*+? — 2) +2 = 2't3 — 2 strings. 


5 Lower Bound for GREEDY on 4-MCSP 


In this section we show that GREEDY’s approximation ratio is (logn) even on 
4-MCSP instances. To simplify the description, we allow the input instances A, B 
to be multisets of equal number of strings, rather than single strings. It is quite 
easy to see that this does not significantly affect the performance of GREEDY, for 
we can always replace A, B by two strings A, B, as follows: If A = {Aj,..., Am} 
and B = {Bi, rare Bm}, let A = AıT1Y1Á2T2Y2 sei Åm-—-1£m-1Ym-1ÁÅm and B = 
Byyi 21 Boyor2... Bm—-1Ym—12m—-1Bm,where z1, Y1,---,;Lm—1,;Ym~1 are new let- 
ters. Then both the optimal partition and the partition produced by GREEDY on 
A, B are the same as on A, B, except for the singletons x1, y1,...,2m—1,Ym-1- 
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Since in our construction m is a constant, it is sufficient to show a lower bound 
of 2(logn) for multisets of m strings. 

For į = 1,2,..., we fix strings qi, qj, Ti, T; that we will refer to as elementary 
strings. Each elementary string q, q}, ri, r} has length 3*~? and consists of 3*~! 
distinct and unique letters (that do not appear in any other elementary string.) 

We now recursively construct instances A‘, B? of 4-MCSP. The invariant of 
the construction is that At, B* have the form: 


At: Pig, Pagiri, Paqi, Pagiri, Psaj, Psairi, Proj, Psairi 
BY: Pigiri, Pog, Paqir;, Pagi, Psairi, Pegi, Preir;, Pegi 


4° 2) 


where P,;,..., Pg are some strings of length smaller than 3*~+ with letters distinct 
from di, Gis Ti, Ti- 
Initially, we set all P4, ..., Ps = €, and construct A}, B} as described above. 


In this case qi, qj, ri, r; are unique, single letters. 

To construct A‘t!, B*+1, we append pairs of elementary strings to the strings 
from A’, B*. For convenience, we omit the subscripts for q and r, writing q = qi, 
q = qi+i, etc. After rearranging the strings, the new instance is 
Art Pyar ġà, Paqr’ gr, Prq'r' G, Pear gr’, Paqr' g, Pzgr qr, Psq'r g, Pedr’ gr’ 
Bt: Pigr gf, Paqr' G, Pra'r’ GF’, Peqd'r G7, Paar’ Fr, Poqr 7, Psq’r 7’, Paq'r' g 

Note that this instance has the same structure as the previous one, since we 
can take Pi = Pygr, P; = Pyqr’, etc., thus we can continue this construction 
recursively. Each letter appears at most four times in A’ and Bt, so this is indeed 
an instance of 4-MCSP; the claimed bound on the length of P; also follows easily. 

Consider now the i-th instance, A* and B*. To estimate the optimal partition, 
we can match the 8 pairs of strings as aligned above, adding the shorter string 
from each pair to the common partition, leaving us with additional 4 strings 7, 
7’, F, 7’, so dist( At, B*) < 12. 

We now show that GREEDY computes a partition with O(i) = O(logn) 
blocks. To this end, we claim that, starting from A**+}, B‘+!, GREEDY first 
matches all suffixes that consist of two elementary strings as shown below (A‘t?} 
and 6*+! are rearranged to show the matched strings aligned vertically): 

Att: Pyar’ ar, Peq'r GF, Pagar 77, Paq’r' FF’, Prard, Prd’ r'ĝ, Poq r'g, Psd rd 
B+. Piar GF, Prq'r’ Gr’, Psqr’ FF, Psq'r 77’, Peq' rg, Paqr’gG, Pad’ r'7, Poa rd 

Indeed, the instance has four common substrings of length 2 - 3*, namely @7, 
qr’, G7, Fr’, and, by the choices of the lengths of elementary strings and the 
bound on the lengths of the P,’s, all other common substrings are shorter. Thus 
GREEDY starts by removing (marking) these four suffixes. Similarly, at this step, 
the new instance will have four common substrings of length 3t + 3*~?, namely 
rg, rg, rq, rq, and all other common substrings are shorter. So now GREEDY 
will remove these four suffixes. The resulting instance is simply A‘, B*, so we 
can continue recursively, getting O(i) blocks. If n is the length (total number of 
characters) of At, we have i = O(log), so the of the lower bound is complete. 
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6 Upper Bound for GREEDY on 2-MCSP Instances 


Consider two arbitrary, but fixed, related strings A = ayjaq---a@n and B = 
byb2---b, 1n which each letter appears at most twice. Let m be a minimum 
common partition of A, B, and denote by g the number of steps of GREEDY on 
A, B. For each t = 0,...,g, let p be the common reference partition of A, B at 
step t, as defined in Section 2. In particular, pp = 7, and pg is the is the final 
partition computed by GREEDY. 

Our proof that GREEDY’s approximation ratio on 2-MCSP instances is at 
most 3 is based on amortized analysis. We show how to define a potential ®; of 
p+ that has the following three properties: 


(P1) Bo < 3 - #blocks(po) + 1, 
(P2) & < Si for t=1,...,g, and 
(P3) B, > #blocks(p,) +1. 


If such ®,’s exist, then, using the optimality of pọ and conditions (P1), (P2), 
(P3), we obtain #blocks(p,) < 4—1 < Bo—1 < 3- #blocks( po) = 3- #blocks(r) = 
3- dist( A, B), and the 3-approximation of GREEDY follows immediately. Thus it 
remains to define the potential and show that it has the desired properties. 


Classification of Breaks. Consider some step t. A break i,t + 1 of p¢ is called 
original if it is also a break of 7; otherwise we call this break induced. Letters 
inside blocks marked by GREEDY are called marked. For any letter a; in A, we 
say that a; is unique in pr if a; is not marked, and there is no other non-marked 
appearance of a; in A. 

Suppose that z,2+ 1 is an original break in py. We say that this break is 
left-mature (resp. right-mature) if a; (resp. a;41) is unique; otherwise it is called 
left-immature (resp. right-immature). If a break is both left- and right-mature, 
we call it mature. If it is neither, we call it immature. (The intuition behind these 
terms is that, if, say, a break i,i + 1 is left-mature, then the value of p:(z) does 
not change anymore as t grows.) We extend this terminology to the endpoints of 
A. For the left endpoint, if a; is unique in p we call this endpoint right-mature, 
otherwise it is immature. Analogous definitions apply to the right endpoint. 


Potential. We first assign potentials to the breaks and endpoints of pz: 


(#1) Each induced break has potential 1. 

(¢2) The potential of an original break depends on the degree of maturity. If a 
break is immature it has potential 3. If it is left-mature and right-immature, 
or vice versa, it has potential 2. If it is mature, it has potential 1. 

(@3) The left endpoint has potential 2 or 1, depending on whether it is right- 
immature or right-mature, respectively. The potential of the right endpoint 
is defined in a symmetric fashion. 


Then #; is defined as the sum of the potentials of the breaks and endpoints 
in pz. For t = 0, all breaks in 7 have potential at most 3 and the endpoints have 
potential at most 2, so we have o < 3- #breaks(m) + 4 = 3- #blocks(x) +1. For 
t = g, all letters in A are marked, so the potentials of all breaks and endpoints 
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are equal 1. Therefore g > #breaks(p,) + 2 = #blocks(p,) + 1. Thus properties 
(P1) and (P3) hold. 

To prove property (P2), first note that the potentials of the breaks of p;_; 
that remain in p; cannot increase. Thus only the new breaks (those in p; but 
not in pz-1) can contribute to the increase of the potential. All these new breaks 
have potential 1 in p+. With each new break we can associate some old break (or 
an endpoint) of p:..; that either disappears in pẹ or whose potential decreases. 
This mapping is not necessarily one-to-one. However, the number of new breaks 
associated with each old break does not exceed the decrease of the potential 
of this old break. This needs some case analysis along the lines of Lemma 2.2; 
details are omitted in this abstract. Summarizing, we obtain the the upper bound 
of 3 for 2-MCSP, i.e., the upper bound in Theorem 1.1(c). 


7 Lower Bound for GREEDY on 2-MCSP 


Let l be a large even integer. Let A’ = aja2...a;2, B’ = biba... biz, and 


ti 
A = @i(1—-1) + 2411-1) 43 -+- G2 +--+  Q@I41Q1+2.--.-Q2X(1—1)+1 0203... QI 
B” = biz bii—=1)+1b1(0—1)+2 e.’ bi2—1 ses bibis1 toe ba(1—1) bibo soe bi—ı, 


where the /? letters a; of A are all distinct, for all i = 1 (mod /+1), b; are new 
letters distinct from each other and from all ay, and b; = a; for all other values 
of i. The instance is A = A'B” and B = A”B’. Obviously, no letter occurs more 
than twice, so this is indeed an instance of 2-MCSP. 

A’ and B’ consist of the same l — 1 substrings of length l, separated (and 
ended) by / distinct markers. The strings A” and B” are obtained from A’ and 
B’ by cutting into l+ 1 substrings of length l — 1 and a single letter, and taking 
these in the reverse order. (Note that the cuts in the two strings are not aligned.) 

The cuts indicated by spaces in the definitions of A” and B” show that 
dist(A, B) < dist(A’, A”) + dist(B’, B") < 2l +4. 

A’ and B’ have l — 1 common substrings of length /, namely the substrings 
between the marker symbols. We claim that A and B have no other common 
substrings of length l. Clearly, by the choice of the markers, A’ and B’ have no 
other common substrings of length /. The longest common substring of A’ and 
A” as well as of B’ and B” has length l — 1, by the definition of A” and B”. 
The strings A” and B” also have no common string of length / or larger, since 
the ends of their substrings of length l — 1 are not aligned. There is no common 
substring that would overlap both A’ and B” or A” and B’. 

Consequently, GREEDY starts by matching the / — 1 common substrings of 
A’ and B’ of length l. After this phase, each letter occurs in the remaining string 
exactly once, and thus the remaining matching is unique. The remaining letters 
a; in A’ with the exception of az are strings of length 1, and they give another 
l — 1 strings in the final partition. Now it is sufficient to bound the number of 
breaks forced in the string B”. There is a break before and after each letter b;, 
i=1 (mod!+1),1<i< J’, as these are strings of length 1 in the remainder 
of B; this gives 2/ — O(1) breaks. There is also a break before and after each 
letter b;, 2 = 1 (mod l -— 1) as for such i, bibi+ı is a consecutive pair in B” 
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but the possibly matching pair a;a;41 is not consecutive in A”; similarly a;_1a; 
is consecutive in A” but b;—;b; is not consecutive in B”. This gives 2l — O(1) 
breaks. Finally, since | ~ 1 and l + 1 are relatively prime, only O(1) breaks 
may be counted twice, by the Chinese remainder theorem. Altogether, GREEDY 
partitions A = A’ B” into 2l — 2 blocks of A’ and 4l — O(1) blocks of B”, for the 
total of 6! — O(1), and the lower bound of 3 on the approximation ratio follows 
by taking l arbitrarily large. 

Final Comments. We have established that GREEDY’s approximation ratio 
is O(n®-®°), but not better than 2(n°-43). It would be interesting to determine 
the exact approximation of this algorithm. In particular, is it below, above, or 
equal to O(,/n)? Also, we have observed a difference between the performance 
of GREEDY on 2-MCSP instances and 4-MCSP instances. Whereas the approx- 
imation ratio for 2-MCSP is 3, for 4-MCSP it is not better than (logn). The 
reason for this is, roughly, that for 2-MCSP every new cut (i.e., induced cut) is 
adjacent to a unique letter and, since GREEDY does not make mistakes on unique 
letters, these new cuts do not induce any further cuts. However, for k > 2, new 
cuts may induce yet more new cuts again. An intriguing question is whether for 
4-MICSP the upper bound matches the 2(logn) lower bound, or whether it is 
higher? The question about exact approximation ratio of GREEDY for kK-MCSP 
remains open even for k = 3. 
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Abstract. We consider the problem of fitting metric data on n points 
to a path (line) metric. Our objective is to minimize the total additive 
distortion of this mapping. The total additive distortion is the sum of 
errors in all pairwise distances in the input data. This problem has been 
shown to be NP-hard by [13]. We give an O(log n) approximation for this 
problem by using Garg et al.’s [10] algorithm for the multi-cut problem 
as a Subroutine. Our algorithm also gives an O(log’ P n) approximation 
for the Lp norm of the additive distortion. 


1 Introduction 


One of the most common methods for clustering numerical data is to fit the 
data to tree metrics. A tree metric is defined on vertices of a weighted tree. 
The distance between two vertices is the sum of the weights of edges on the path 
between them. Here the main problem is to find a tree metric that represents the 
input numerical data as closely as possible. This problem, known as numerical 
taxonomy, has applications in various fields of science, such as linguistics and 
evolutionary biology. For example, in evolutionary biology tree metrics represent 
the branching process of evolution that leads to the observed distribution of data. 
Naturally, this problem has received a great deal of attention. (e.g. see [3,14]). 
The problem of fitting data to tree metrics is usually cast as the problem of 
minimization of L,(D,T): the Lp norm of additive distortion of the output tree 
T with respect to input data D. The input data is specified as ann x n matrix, 
where the entry D;; denotes the distance between points 2 and j. Let Tj; denote 
the distance between 7 and j in the output tree metric T. Then the Lp norm 
of additive distortion is L,(D,T) = (0, ;|Dis - T;;|?)!/?. Such a formulation 
was first proposed by [5] in 1967. In 1977, Waterman et al. [15] showed that if 
there is a tree metric T coinciding exactly with the input data D, then it can be 
constructed in linear time. In the case when there is no tree that fits the data 
perfectly, Agarwala et al. [1] used the framework of approximation algorithms 
to give heuristics with provable guarantees for the problem. They gave a 3- 
approximation to the ZL. norm of the additive distortion for fitting the data to 
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a tree metric. They reduced the problem to that of fitting the data to ultrametric, 
where each leaf is at the same distance from a common root. For ultrametrics, 
they used an exact polynomial-time algorithm for the Leo norm due to Farach et 
al. [8]. Agarwala et al. [1] showed that if there is a p-approximation algorithm 
for ultrametrics under the Lp norm, then it implies a 3p-approximation for tree 
metrics under the Lp, norm. Our work is motivated by this problem. 

For a special case of fitting the data to a tree metric under Lı norm, we 
make a simple observation. Suppose we know the structure of the tree along 
with a mapping of the points to the vertices of the tree. Then we can find the 
edge weights that minimize the Lı norm of additive distortion using linear pro- 
gramming. However, finding the topology of the tree is the hard part. Therefore, 
we consider a special case of the problem in which we restrict the structure of 
the output tree to be a path. For this special case, we give an approximation 
algorithm. We believe that our techniques can be extended to solve the original 
problem. The main result of this paper is the following theorem. 


Theorem 1. There is an O(log’ Pn)-approzimation algorithm for the problem 
of fitting metric data to a line metric to minimize the L, norm of additive 
distortion for p > 1. 


1.1 Related Work 


Note that fitting points to a path metric is equivalent to fitting points on a 
real line, where the distances in the real line are defined in a natural way. The 
special case of the problem for the Loo norm (i.e. with p = oo) was considered 
by Hastad et al. [11]. They gave a 2-approximation for it. 

For fitting points to a line, a well-known result due to Menger (see e.g. [6]) 
gives the following four point criterion. The four point criterion says that, if 
every subset of size 4 can be mapped into the real line exactly, then all the 
points can be mapped into the line exactly. An approximate version of Menger’s 
result was given by Badoiu et al. [2]. They proved that if every subset of size 
4 can be embedded into the line with the Leo norm of the additive distortion 
being at most € then all the points can be embedded with the Leo norm of the 
additive distortion being at most 6e. 

In a related work, Dhamdhere et al. [7] and Badotu et al. [2] independently 
studied average distortion of embedding a metric into path metrics. The objective 
was to minimize }7,, Jij, Subject to fi; > Di; for all i,j, where Dj; is the 
distance between points 2 and 7 in the input and f;; is the distance between 
them in the output path metric. They gave a 14-approximation for the problem. 
While their problem has additional constraint f;; > Diz, their objective function 
is easier than the Lı norm of the additive distortion. We do not know how to 
minimize the Lp (or even the Lı) norm of additive distortion under the additional 
constraint. However, for the special case of p = oo, we would like to point out 
that the algorithm for min-excess path problem due to Blum et al. [4] gives a 
2 +€ approximation. The min-excess path problem asks for a path from a source 
s to a destination t that visits at least k vertices and minimizes the objective 
(path) — d(s,t),where l(path) is the length of the path. 
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1.2 Techniques and Roadmap 


In Section 2, we define our problem formally. In Section 3, we show how to reduce 
the problem to that of finding the best r-restricted mapping. An r-restricted 
mapping of the input points into a line is one in which the distances of all 
points in the line from point r are same as that in the input. We show that this 
problem is equivalent to a two-cost partition problem. In Section 4, we give an 
approximation for this problem via the multi-cut problem [10]. 


2 Problem Definition 


Consider a set of n points, denoted by [n] = {1,2,..., n}. The input data consists 
of an n x n matrix Dnxn. The entry Di; denotes the distance between points 
i and 7. We assume that all the entries of D are non-negative and that D is 
symmetric’. Furthermore, we assume that D,; = 0 for all i. 

Let f : [n| — R denote a mapping of the input points to the real line. 
Distance between images of points 2 and 7 in the line is given by fi; = |f(z) — 
f(j)|. The total additive distortion (in the Lı norm) is given by 


Lı(D, f) = > lD = feh 


Generalizing this, we can write the Lp norm of the additive distortion as 


i 
P 


L,(D, f) m 2 |Di; a Fij|?) . 


The goal is to find a map f that minimizes the Lı(D, f) (or more generally 
Lp (D ’ f ))- 


3 Approximation for L, Norm 


In this section, we give an approximation algorithm for minimizing the Lp norm 
of the additive distortion. 

In Lemma 1, we will show that it is sufficient to look at r-restricted mapping 
of the points into the real line. The problem of finding an optimal r-restricted 
mapping can be cast as a kind of partition problem given the characteristics of 
the real line. 


3.1 r-Restricted Mappings 


Let r be a point in the input. A mapping f of the input points to the real line 
IR is an r-restricted mapping, if distance on the line of all points from r is same 
as that in the input. Formally, Dr; = |f(r) — f(| for all 7. 


' Our results hold even if the input distances in D,,~» do not satisfy triangle inequality, 
i.e. even if D is not a “metric”. 
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We will denote an r-restricted mapping by f”. We next show that there is 
always a“good” r-restricted mapping. This will enable us to focus only on r- 
restricted mappings which are easier to handle. Agarwala et al. [1] prove a similar 
lemma for tree metrics. We adapt their proof for the case of line metrics. 


Lemma 1. There exists a point r among the input points such that there is an 
r-restricted mapping f” that is within a factor of 3 of the optimal mapping for 
the L,norm of additive distortion, forall p> 1. 


Proof. Let f* denote an optimal mapping of the input points to the line for the 
L, norm of additive distortion. We will modify the optimal solution to produce 
a mapping f* for each point i in the input. To produce the restricted mapping 
ft, perturb the distances in f* so that it becomes i-restricted. In particular, if 
f*(3) < f*(@) forsome j, then set fi) = f*(t) — Dj; and if f*(7) > f*(2), set 
f°) = f*(@) + Dij. Our mapping f* maps point i to f*(i). It maps rest of the 
points according to their distance from 2, while maintaining their order to the 
left or right of point 7 in the optimal mapping f*. 

Let €;~ denote [D;k — f7,|. We can write the additive distortion of the optimal 


mapping as Lp(D, f*) = (05% oe P. From the construction of the map ff, it 
follows that |f}, — firl < €ij + €ik. 

Now we bound the additive distortion of f* in terms of €;,’s. For all j,k we 
have, 


|\Dik — re Se kl + [Dik — fik! 
< (€i; + Eik) + Ejk (1) 


Note that |x|? is a convex function of x for p > 1. Therefore, Equation (1) 
gives us the following: 


|Dik — Fe < (eij + Eik + Ejk)” 
< BPN (e + eik + Gu) (2) 


By an averaging argument, we can say that 


min{Lp(D, f*)"} < mee y 


We use Equation (2) to bound the sum 


ve) Te 
2 loD, FY sD DP ay + he + Gi) 
t=1 


i=1 j,k 
j,k 


= 3?n- Lp(D, f*)? 


A 


Therefore, min; Lp(D, f*) < 3- Lp(D, f*) which proves the result. 
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3.2 Algorithm 


The result of Lemma | proves that it is sufficient to consider r-restricted map- 
pings (with a loss of 3 in the approximation factor). Next we describe the algo- 
rithm that implements this idea. 


Algorithm A: 


l. For each point r = 1,2,...,n, find (approximately) the best r-restricted 
mapping f”. 

2. Output a mapping that has the smallest additive distortion among these 
mappings. 


By Lemma 1, the additive distortion of the output of Algorithm A is within 
a factor of 3 of the optimal additive distortion. As we point out in Section 5, 
finding best r-restricted mapping is NP-hard. Therefore, we approximate the 
optimal a-restricted mapping within a factor of O(log!” n). From the following 
observation it follows that the overall approximation factor of our algorithm will 
be O(log? n). 


Lemma 2. Ifp is the approximation factor of the algorithm for r-restricted 
mapping, then the solution produced by Algorithm A will be a 3p approximation 
for the additive distortion. 


4 Approximating r-Restricted Mappings 


Let f be an r-restricted mapping. Without loss of generality, we can assume 
that f(r) = 0. Let Vi = {i | f(z) < 0} and Vo = {i | f(z) > 0}. Note that 
[In] = Vi U {r} U V2. Note that, the mapping f is fully characterized by the 
partition Vi U Vz of [n] — {r}. Hence, the problem of finding the best r-restricted 
mapping is equivalent to the problem of finding the partition of V = [n]~{r} that 
has minimum additive distortion. Henceforth, we will think of the problem as 
that of partitioning the input set of points to minimize the cost of the partition, 
1.e. the additive distortion. For simplicity, we describe the argument for p = 1. 
The other cases (p > 1) are similar. 

Consider a partition V; U V2 induced by an r-restricted mapping f. We can 
write an expression for its cost as follows. Consider two points x and y. If they 
both belong to the same side of the partition, then the contribution of the pair 
{x,y} to the cost of the partition is e(z, y) = |Dzy — fry| = Dey —|f (x) — f(y) = 
[Dey —|Drz — Dryll. On the other hand, if x and y belong to different sides of the 
partition, then the contribution is ¢’(z, y) = |Dzy— fry| = |Dzy—|f(x)—Ff(y)|] = 
|Dra + Dry —Day|. Note that e(z, y) and e’(x, y) are completely determined from 
the input matrix Dnxn. 

Therefore, we can think of the problem as a graph partitioning problem where 
each edge has two costs c(-) and e’(-) associated with it. The objective function 


is 
>D c(x, y) + bD e'(z,y) 


x,y on same side x,y on different sides 
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4.1 Two-Cost Partition Problem 


We are given a complete graph G = (V, E) with two cost functions ec and œ. 
We want to find a partition of the vertex set V = Vi U Va which minimizes 
2 si=1,2 Dou,vev, CU: V) + diuevy,veVv, C (u,v). 

Note that, if c(u,v) = 0 for all u,v, then the problem reduces to finding a 
minimum cut in the graph. On the other hand, if c’(u, v} = 0, then the problem 
is the well known edge deletion for graph bipartition problem (BIP) [12]. Our 
algorithm generalizes the algorithm for graph bipartition given by [12,10]. The 
basic idea is to create two copies of each vertex to go on different sides of the 
partition. To ensure that they are on different sides, we designate each pair as a 
source-sink pair in the multi-cut subroutine. 


Algorithm B: 


1. Create an auxiliary graph G’ from the graph G as follows. 
(a) For each vertex u in the graph G, G’ has two vertices: u and w’. 
(b) For each edge (u,v) we create 4 edges in G’: (u,v), (u, v’), (u’,v) and 
(u’,v’). 
(c)The edges in G’ have weights, denoted by l(.,-). Set l(u,v) = [(u’,v’) = 
c(u, v) and l(u,v’) = l(u', v) = e'(u, v). 

2. Use an approximation algorithm for the multi-cut problem (E.g., [10]) as 
a subroutine to find a multi-cut in graph G” with (u,u’), for all u, as the 
source-sink pairs. Let S be the set of edges in the multi-cut returned by the 
subroutine. 

3. Construct a set of edges T as follows. If {u,v} or {u’,v’} is chosen in S, then 
include both in T. Similarly, if {u,v} or {u’, v} is chosen, then include both 
in T. 

4. Find a bipartition Vf U VJ of vertices of G’ so that T contains all the edges 
going across the partition’. 

5. Output the partition Vi U V2, where V; = V/ NV. 


The intuition behind this algorithm is as follows. For the cut represented by 
T, we will show that we can get a partition of vertices in graph G’ such that only 
one of u and wu’ is in one partition. From the partition of G’, we get a bipartition 
of G. The cost of the bipartition of G is related to the cost of multi-cut obtained 
by above algorithm in the graph G’. We prove this in the next lemma. 


Lemma 3. Algorithm B returns a partition V' = V{UV3 of graphG", such that 
if u € V}, then u' € Vy and vice versa. Moreover, a xeV; yeV; (x,y) is at most 
twice that of the multi-cut found after step 2 by Algorithm B separating each u 
from w. 


Proof. Consider the set S of edges found by the multi-cut subroutine whose 
removal separates each u from u’. For each edge (x,y) € S, we also include its 
“mirror” edge in T. i.e. if (x,y) € S, then (z’,y’) € T from the graph. Note 


? We will show how to do this in the proof of Proposition 1. 
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that, the cost of an edge and its “mirror” edge is same (i.e., I(x, y) = I(x, y’)). 
Therefore, the cost of the edges in T is at most twice the cost of edges in S. 

Now we show that removal of the edges in T breaks the graph in two parts 
with the desired property. Consider the graph G’\T.. Construct a graph H whose 
vertices represent the connected components in G’ after removing the edges in 
T. Two vertices hı and hg in H are connected to each other if the corresponding 
connected components in G” have vertices x and z’. 

In Proposition 1, we prove that the graph H is bipartite. Now we can use 
graph H to construct a partition V’ = Vf U V3 in graph G”. Since the vertices 
in graph H were connected components in graph G’, there are no edges crossing 
the partition Vf U V3 in graph G’. Moreover, bipartiteness of graph H means 
that each pair of vertices x and x’ in graph G is split in the partition. The cost 
of this partition is at most 2 times the cost of the multi-cut. 


Proposition 1. The graph H defined in the proof of Lemma 3 is bipartite. 


Proof. For the sake of contradiction, assume that H has a cycle of odd length. 
Consider three consecutive vertices u,v and w in this odd cycle. Let v be con- 
nected to u and w. 

Let x be a vertex of G” that belongs to the connected component u and defines 
the edge {u,v} in graph H. Therefore, x’ is the component v. Similarly, let y be 
a vertex in component w and y’ be the corresponding vertex in component v. 
Since x’ and y’ are in the same connected component v, there is a path x’ — y’ 
that lies completely inside the component v. Since we didn’t remove any of the 
edges on the path x’ — y’, all the mirror edges haven’t been removed either. 
Therefore the the mirror path x — y connects x and y. This contradicts the fact 
that x andy were in different connected components. 

This proves that the graph H is a bipartite graph. 


Lemma 4. The cost of the optimal multi-cut is a lower bound on the cost of 
partition of graph G. 


Proof. Consider a partition V = V1 U Vz of graph G. From this, we can construct 
a partition of the vertex set of G’. Let VI = V, U{a2’ | z € Vo} and Vy = V'\V7. 
Then, removing all the edges in G” crossing this partition ensures that no vertex x 
is connected to its counterpart x’. i.e. The set of edges going across the partition 
is a multi-cut. The cost of these edges is exactly the cost of the partition of G. 


Recall that GVY algorithm for multi-cut [10] is an O(log k) approximation 
for k terminals. Here we have n terminals. Therefore by Lemmas 3 and 4, we 
get an O(logn) approximation for the best r-restricted mapping. Along with 
Observation 2 give us an O(logn) approximation for the Lı norm of additive 
distortion. 

To get an approximation algorithm for the Lp norm, we modify the costs in 
the two-cost partition problem as follows. Let e(z, y) = |Dz,y—|Daz — Day||? and 
d(x,y) = |Das + Day — Day|?. With these costs, Algorithm B gives an O(log n) 
approximation for L,(D, f*)?. Therefore, for the Lp norm of additive distortion, 
we get an O(log!” n) algorithm. 
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5 Open Questions 


We can show that the problem of finding the best r-restricted mapping is NP- 
hard by reducing the edge deletion for graph bipartition (BIP) [9] to it. Consider 
a graph G. Let V(G) = n. We construct a distance matrix D on n + 1 points 
V(G) U {a}. Set the diagonal entries Dzs to 0. Set Daz = 1/2 for all z € V(G). 
For all {x,y} € E(G), set Dz, = 1. Set the rest of the entries to 1/2. Consider 
an r-restricted mapping. Let V(G) = Vi U Vz be the partition induced by the 
r-restricted mapping. Then the cost of the r-restricted mapping is B(V;, V2) + 
(1/2)((5) — |E(G)|), where B(V1, V2) is the number of edges that need to be 
deleted to obtain V; and Vz as two sides of a bipartite graph. Therefore, finding 
the optimal r-restricted mapping corresponds to minimizing the number of edges 
deleted for making the graph G bipartite. This proves that finding the best r- 
restricted mapping is NP-hard. However, this reduction is not approximation 
preserving. So it does not preclude the possibility of a PTAS for this problem. 
Getting even a constant factor approximation would be quite interesting. 

In the proof of NP-hardness of r-restricted mapping problem, we used an 
input matrix D that does not satisfy the triangle inequality. For input matrix D 
that is a metric (i.e. it satisfies the triangle inequality), it might be possible to 
get a polynomial time algorithm for the best r-restricted mapping. 

Agarwala et al. [1] have shown that for the problem of fitting data to tree 
metrics, the best r-restricted tree metric is within a factor of 3 of the optimal 
solution. However, the problem of approximating the additive distortion of the 
best r-restricted tree is still open. 
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Abstract. We prove that there exists a universal constant c > 0 such 
that the Radio Broadcast problem admits no additive c- log? n-approxi- 
mation, unless NP C BPTIME(n?0°8!°6")), For graphs of at most 
logarithmic radius, an O(log? n) additive approximation algorithm is 
known, hence our lower bound is tight. To the best of our knowledge, 
this is the first tight additive polylogarithmic approximation result. 


1 Introduction 


1.1 The Radio Broadcast Problem 


Definition and Motivation. Consider a synchronous network of processors 
that communicate by transmitting messages to their neighbors, where a proces- 
sor receives a message in a given step if and only if precisely one of its neighbors 
transmit. The instance of the Radio Broadcast problem, called radio network, is 
a pair (G = (V, E), s), s € V, where G is an unweighted undirected graph, and s 
is a vertex, called source. The objective is to deliver one single message that the 
source s generates to all the vertices of the graph G using the smallest possible 
number of communication rounds. The prescription that tells each vertex when 
it should broadcast is called schedule; the length of the schedule is the number 
of rounds it uses, and it is called admissible if it informs all the vertices of the 
graph (see Section 2 for a formal definition). From practical perspective, the 
interest to radio networks is usually motivated by their military significance, as 
well as by the growing importance of cellular and wireless communication (see, 
e.g., [KM-98,GM95,BGI91]). The Radio broadcast is perhaps the most impor- 
tant communication primitive in radio networks, and it is intensively studied 
starting from mid-eighties [CR-03,KP02,KP-03,CGR-00,CG+02,CMS-03,I-02] 
[GM95,KM-98,ABLP91,BGI91,CW-87,CK-85]. 


“ Part of this work was done while the first author was in School of Mathematics, 
Institute for Advanced Study, Princeton, NJ, USA, 08540. 
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From theoretical perspective, the study of the Radio Broadcast problem pro- 
vided researchers with a particularly convenient playground for the study of 
such broad and fundamental complexity-theoretic issues as the power and lim- 
itations of randomization, and of different models of distributed computation 
[BGI91,KM-98,KP-03]. In this paper we study the additive approximation thresh- 
old of the Radio Broadcast problem; we believe that our results show that this 
problem is of a particular interest from the stand-point of the theory of Hardness 
of Approximation as well. 


Previous Results. The first known algorithm for the Radio Broadcast problem 
was devised by Chlamtac and Weinstein in 1987 [CW-87]. That algorithm, given 
an instance (G,s) of the problem, constructs an admissible broadcast schedule 
of length O(Rad(G, s) -log* n) for this instance, where Rad(G, s) stands for the 
radius of the instance (G, s), that is, the maximum distance dg(s, v) in the graph 
G between the source s and some vertex v € V. Their algorithm is centralized, 
1.e., it accepts the entire graph as input. 

Soon afterwards Bar-Yehuda et al. [BGI91] devised a distributed random- 
ized algorithm that provides admissible schedules of length O( Rad(G, s) -log n+ 
log? n). Alon et al. [ABLP91] have shown that the additive term of log? n in 
the result of [BGI91] is inevitable, and devised a construction of infinitely many 
instances (G, s) of constant radius that satisfy that any broadcast schedule for 
them requires (log? n) rounds. Kushilevitz and Mansour [KM-98] have shown 
that for distributed algorithms, the multiplicative logarithmic term in the re- 
sult of [BGI91] is inevitable as well, and proved that for any distributed algo- 
rithm for the Radio Broadcast problem there exist (infinitely many) instances 
(G,s) on which the algorithm constructs a schedule of length 2(Rad(G, s) - 
log(n/Rad(G, s))). Finally, the gap between the log n and log(n/Rad(G, s)) was 
recently closed by Kowalski and Pelc [KP-03], and Czumaj and Rytter [CR-03]. 

Gaber and Mansour [GM95] have shown that centralized algorithms are ca- 
pable of constructing much shorter schedules, and devised an algorithm that 
constructs admissible schedules of length O(Rad(G, s) + log’ n). Their result is 
also the current state-of-the-art in terms of the existential upper bounds on the 
length of broadcast schedules for instances of large radius. 


Remark: In [EK-03] we have improved the result of [GM95] by providing sched- 
ules of length Rad(G, s) + O(,/Rad(G, s) «log? n) = O(Rad(G, s) + logt n). 

Since, obviously, any schedule for an instance (G,s) requires at least 
Rad(G,s) rounds, the algorithms for the Radio Broadcast problem 
[CW-87,BGI91,GM95,KP-03,CR-03] can be interpreted as approximation algo- 
rithms for the problem. The state-of-the-art in this context are the results by 
Bar-Yehuda et al. [BGI91] for instances of small radius, and the result of Gaber 
and Mansour [GM95] for instances of large radius. (We term such an approxi- 
mation guarantee “a semi-additive”’, as opposed to “a purely additive” guarantee 
for which the multiplicative factor of the optimum should be equal to 1.) 

In terms of the lower bounds, the NP-hardness of the Radio Broadcast prob- 
lem was shown by Chlamtac and Kutten [CK-85] already in 1985. The authors 
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of the current paper have shown [EK-02] that there exists a constant c > 0 
such that the Radio Broadcast problem cannot be approximated with a (purely) 
additive approximation ratio of clogn unless NP C BPTIM E(n0(8!087)), 


1.2 Our Results 


In this paper we show that there exists a constant c > 0 such that the Radio 
Broadcast problem cannot be approximated with a (purely) additive approxi- 
mation ratio of clog? n, unless NP C BPTIME(n0(e!”)). This result holds 
even for the Radio Broadcast problem restricted to graphs of constant radius. We 
believe that our reduction contains lower bound ideas for radio model context 
that should have further applications. 

Observe also that our results together with [BGI91] fully determine the ad- 
ditive approximation threshold (up to constants and with randomization al- 
lowed) of the Radio Broadcast problem restricted to instances (G,s) of radius 
Rad(G, s) = O(logn). 

Hence we have demonstrated that the approximation behavior of the Ra- 
dio Broadcast problem is best-described in terms of an additive polylogarith- 
mic threshold. It appears that there is no other problem whose approximation 
threshold is known to exhibit a similar behavior. We believe that demostrating 
the existence of a new pattern of approximation behavior enriches the theory of 
Hardness of Approximation, and constitutes the main contribution of this paper. 


1.3 Proof Techniques 


The proof of our lower bound incorporates the combinatorial construction of 
Alon et al. [ABLP91] in the reduction of Lund and Yannakakis [LY94]. The 
latter reduction reduces the Label-Cover problem to the Set Cover problem, and 
establishes the logarithmic hardness of approximation of the latter. However, 
the importance of this reduction extends far beyond this specific proof, and 
the reduction was used as a building block for many inapproximability results 
[ABSS93,EK-02, DGKRO03,HK-03]. 

However, it seems that the reduction of [LY94] alone cannot provide greater 
than logarithmic for the Radio Broadcast problem. The second crucial building 
block of our lower bound is the result of Alon et al. [ABLP91]. Their construction 
provides a bipartite graph in which it requires 2(log? n) rounds to deliver the 
message from the left-hand side of the graph to its right-hand side. 

To explain how we incorporate the construction of [ABLP91] into the re- 
duction of [LY94], we first outline the structure of the reduction of [LY94]. For 
each edge é of the input instance of the Label-Cover problem, the reduction of 
[LY94] constructs a bipartite graph (Sz, Mz), where each vertex x € Sz is con- 
nected to one half of the vertices of the set Mz. Furthermore, these connections 
are ingeniously arranged in such a way that the following condition is satisfied: 
for any k = o(logn) vertices 11,%2,...,2% E Sz, and for any choice of sets 
X1, X2,..., Xx such every X; is either equal to the set N; of the vertex z; in Me 
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or to its compliment M;\Nj, the set Ua X; does not cover the vertex set Me. 
Such a bipartite graph is called a set system. 

The reduction of [LY94] also provides each such set system (Sz, Mz) with a 
simple “trapdoor” that enables to cover the set Ma with only two sets, specifically, 
the set N; and the set Mz\N;. These trapdoors are geared to enable to construct 
small solutions for the instances of the Set Cover problem whenever the original 
instance of the Label-Cover problem is a YES-instance, but, however, cannot be 
used if the original instance is a NO-instance. 

The basic idea of our proof of the polylogarithmic lower bound for the Ra- 
dio Broadcast problem is to replace the set systems of [LY94] by the bipartite 
graphs of [ABLP91]. To accomplish it, one, however, needs to devise analogous 
trapdoor to the construction of [ABLP91], and doing it without ruining the 
(rather delicate) properties of [ABLP91]. It 1s, therefore, not surprising that our 
construction is quite complex, and its analysis is technically very involved. 

Comparison with [EK-02]: The reduction of [EK-02] can be seen as a (non- 
trivial) adaptation of the reduction of Lund and Yannakakis to the Radio Broad- 
cast problem. The reduction in the current paper is far more complex both con- 
ceptually and technically, particularly because it incorporates the construction 
of [ABLP91] in the reduction of Lund and Yannakakis. 


2 Preliminaries 


2.1 The Radio Broadcast Problem 
We start with introducing some definitions and notations. 


Definition 1. The set of neighbors of a vertex v in an unweighted undirected 
graph G(V,E), denoted I'g(v), is the set {u € V | (v,u) € E}. For a subset 
X C V, the set of neighbors of the vertex v in the subset X, denoted I'g{v, X) 
(or I'(v,X) when the graph G is clear from the context), is the set{u € X | 
(v,u) € E}. 


Definition 2. Let G = (V,E) be an unweighted undirected graph, and RC V 
be a subset of vertices. The set of vertices informed by R, denoted I(R), is 
I(R) ={v| alae R s.t. v€Ie(x)} (the notation atx stands for “there exists 
a unique x”). For a singleton set R = {x}, I(R) = I({x}) = I(x) = Te(z). 


A sequence of vertex sets I = (Ri, Ro,...,R,), q = 1,2,..., is called a radio 
broadcast schedule (henceforth referred as a schedule) if Ri+ı C oa I(.R;) for 
every i = 1,2,...,q—1. The set of vertices informed by a schedule IT, denoted 
(II), is (UT) = Uren I(B). n a 

An instance of the Radio broadcast problem G is a pair (G = (V,£),s), 
where G is a graph, and s € V is a vertex. The goal is to compute an admissible 
schedule IZ of minimal length. The value of an instance G of the radio broadcast 
problem is the length of the shortest admissible schedule JJ for this instance. 
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2.2 The MIN-REP Problem 


Definition 3. The MIN-REP problem is defined as follows. The input consists 
ofa bipartite graph G = (V1, V2, E). In addition, for 7 = 1,2, the input contains a 
partition V; of V; into a disjoint union of subsets, Vi = yey, A, V2 = Ugen B- 
The triple M = (G, Vi, V2) is an instance of the MIN-REP problem. The size 
of the instance is n = |Vi| + |V2|. An instance G as above induces a bipartite 
super-graph G = (Vi, Vo, E) in which the sets A and B of the partition serve 
f the vertices of the super-graph. The edges of the super-graph are E(M) = 

E = {(4, B) € Vi x Vz | a € A,b € B, (a,b) € E}. In other words, there is a 
(super-)edge between a pair of sets A € Vi, B € Vz if and only if the graph G 
contains an edge between a pair of vertices a € A, b € B. 

Denote V = Vi U Vz. A pair of vertices z1, £2 € Vi U Vz is called a match- 
ing pair with respect to a super-edge € = (A, B) € E (henceforth, è-m.p.) if 
(z1, £2) € E and either x, € A and z3 € B or vice versa. 

A subset C C Vi U V2 of vertices is said to cover asuper-edge € = (A, B) if 
it contains an é-m.p.. A subset C C Vi U Və that satisfies |C N X| = 1 for every 
X eV is called a MAX-cover. In other words, a MAX-cover C contains exactly 
one vertex from each super-vertex. An instance M of the MIN-REP problem is 
called a YES-instance if there exists a MAX-cover that covers all the superedges. 
Such a MIN-REP solution is called a perfect MAX-cover. For a positive real 
numbert > 1, an instance M ofthe MIN-REP problem is called at-NO-instance 
if any MIN-cover C of the instance M covers less than |E|/t super-edgesof M. 


We also impose several additional (somewhat less standard) restrictions on 
the set of instances of the MIN-REP problem. 


1. All the super-vertices X € V are of size |X| = n°-4/2. 

2. The set of super- edges has sufficiently large cardinality, specifically, |E| = 
2(n*® - log’ n). (Note that by restriction 1, just for the super-graph to be 
connected, |E] has to be at least 2n°-® — 1.) 

3. The Star property: For every super-edge ë = (A,B) € E, A C Vi and 
B C V and every vertex b € B there exists exactly one vertex a € A, 
denoted é(b), such that (a,b) € E. This property is called the star property. 


Theorem 1. /R98] No deterministic polynomial time algorithm may distinguish 
between the YES-instances and the log'®° n-NO-instances of the MIN-REP prob- 
lem, unless NP C DTIME(n?8!8™)) | even when the instances of the MIN- 
REP problem satisfy the conditions (1)-(3). 


3 Reduction 


We next describe our randomized reduction from the MIN-REP problem to 
the Radio broadcast problem. This reduction always maps a YES-instance of 
the MIN-REP problem to an instance of the Radio broadcast problem of value 
T + O(log n), and with high probability over its own coin tosses the reduction 
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maps a NO-instance of the MIN-REP problem to an instance of the Radio broad- 
cast problem of value T + (log? n), where T = 1,2,... is a parameter of the 
reduction. a 

Consider an instance M = (G, Vi, V2) G = (Vi, V2, E) of the MIN-REP 
problem with Vj = U,cy, A, Ve = U Bev, B. The reduction constructs an in- 
stance G = G(M) = (G,s), G = (V, E), s € V, of the Radio broadcast problem 
in the following way. 

Let N = n®ê. The vertex set V of the graph consists of the source s, and 
the disjoint vertex sets V, and Vz (i.e., V = {s} UV, U Vz with s¢V, U Vz, and 
Vi N Vz = Ø). It is convenient to visualize the source s on the “left-hand” side 
of the graph, the vertex set V} in the “middle”, and the vertex set V2 on the 
“right-hand” side of the graph. The vertex set Yı contains N copies of every 
vertex s € V = V, UV); these copies are denoted cp,(x), cpo(z),..., cpp (2). 
The set of all copies of a vertex x, {cp,(x), cpo(x),...,cpy(z)}, is denoted by 
Xz. For a subset X G V, let xx denote xx = Shee x Xz. Let J denote the set of 


indices {0.4logn,0.4logn4+1,...,0.6logn}. For an index j € J, and a vertex 
xz € V, let yz(7) denote the subset of the set x, that contains only the first 
J = 2) copies of the vertex z. I.e., Xe(j) = {cp; (£), epo(x),..., cp z(x)}. These 
copies are also called the j-relevant copies of the vertex x. For a subset X C V, 
let xx(j) = Usex Xx (J). 

The vertex set Vo is of the form Vz = Use ë Me, where the ground sets Mz 
are disjoint, and all have equal size. Each ground set Mg is a disjoint union of 
the sets Mz(j,q), j € J, q € [L], with L = 2-n™, and co is an integer positive 
universal constant that will be determined later. The sets Mz(j,q) are all of 
equal size M = n™, for the same constant co. 

The edge set Æ of the graph G contains edges that connect the source s to the 
vertices of the set Vj, and edges between the vertices of Vı and V9. Specifically, 
the edge set E is formed in the following way. 


1. Connect the source s to all the vertices of the set V1. 
2. For every super-edge € = (A,B) € E, for every pair of indices j € J, q € [L] 
do 


(a) For every vertex a € A 


i. Let Hea(j,q) be an exact random half of the set Mz(j, q). 
In other words, every subset H C Mag(j,q) that has cardinality 
|Mz(j,q)|/2 (for simplicity we ignore all the Integrality issues) has 
the same probability to be chosen to serve as Hz a(j, q). 


ii. Let H? (j,q) be an exact random half of the set Hë a(j, q). 
Set HP (3,4) = Haa (j, D\HE (G, 9). 
(b) i. For every vertex b € B, set Ha a(j, q) = Meli, q)\ Hz ew (3, q). 
ii. Let HM, b(j,q) be an exact random half of the set Hg »(j, q). 


Set HE? (j, q) = Halj, a) HSR G, a). 
(The steps 2a-2b will be referred as the exact partition step.) 
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(c) For every vertex x € AUB let xP G) be a subset of Xz(j) that is 
formed by picking each element of the set xz(j) with probability 1/2 
independently at random. 

Set x2 (J) = xe(4)\xe” (I). 
(The step 2c will be referred as the binomial partition step. 

(d) For every vertex x € AU B, and index i € {1,2} do 


Let x2) (9) = {ep}? (2,3), ery (2,9), -++5 PY’ (2, J)}, with X = X(z, 4,4) 
= |x G): 

Let Sz be the set of all permutations of the set [X]. 

Choose a permutation o Er Sy uniformly at random from the set Sy. 
Partition the set H Se (j,q) arbitrarily into % disjoint subsets re (jq, 2), 
leE [x]. 

For every index £ € [x], connect the vertex cp? (x,j) to every vertex v 
in the set Ps’ (j,q,0(8)). 

(The step 2d will be referred as the random permutation step.) 

(e) i. For every copy cp(x) € x ( j), and every vertex v € H (j q), with 
probability 1/J insert the edge (x, v) into the edge set Ẹ, indepen- 
dently at random. 

ii. Do the same for every copy cp(x) € x? (j), and every vertex v € 
Ms), q)\He,2(J, q). 

iii. Do the same for every copy cp(x) € X\Xz(j), and every vertex 
VE Me( ĵ, q). 

(The step 2e will be referred as the mixing step.) 


We need some additional notation. For a pair of vertices x € Vy and v € 
Vo, let (x C v) denote the event ((z,v) € E), and let (x M v) denote the 
complimentary event ((z,v)¢E). For a subset X C VY, and a vertex v € Vy, let 
(X AC v) denote the event (Vz € X, (z,v) € E), and, analogously, let (X AN v) 
denote the event (Yx € X, (x,v)¢E). For a pair of subsets X C V1, U C Va, let 
(X AC U) denote the event (Vz € X,v € U, (x,v) € E), and, analogously, let 
(X AN U) denote the event (Vz € X,v € U, (2,v)¢E). 

For a subset X C Vı, a vertex v € Vo, and a positive integer £ = 1,2,..., let 
(X (> 4C) v) denote the event (|{z € X | (z,v) € E}| > 2). Analogously, let 
(X (< £C) v) denote the event (Hx € X | (x,v) € E}| < £), and let (X (£C) v) de- 
note the event ({x € X | (x, v) € E}| = £). We also use the notation (X INF v) 
for the event (X (1C) v); in this case we say that the subset X informs the vertex 
v. For the complimentary event we use the notation (X NZINF v), and say that 
the subset X does not inform the vertex v. For a subset U C Yz, we use the 
notation (X TNF U) for the event (Vu € U, (X INF v)); in this case we say 
that the subset X informs the subset U. For the complimentary event we use the 
notation (X NMIN F U), and say that the subset X does not inform the subset 
U. For a collection of subsets H = (X1, X2,..., X+) of V; and a vertex v € Vo 
(respectively, a subset U C Vz), we say that the collection JI informs the vertex 
v (resp., the subset U), and denote it by (II INF v) (resp., (I INF U)), if at 
least one subset X € IT informs v (resp., U). For the complimentary event we 
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use the notation (I NIN F v) (resp., (I NINF U) and say that the collection 
IT does not inform the vertex v (resp., the subset U). 

For a fixed super-edge é = (A, B) € E, and a vertex z € AUB, let star(z) 
denote the subset of vertices of A U B that belong to the same star as x with 
respect to the super-edge ê. For a subset S C AU B, let cp(S) denote the set of 
all the copies of the vertices of S, i.e., ep(S) = {cp,(x) | x € S}. 


4 Analysis 


In this section we analyze the reduction described in the previous section. 


4.1 Basic Properties 


Definition 4. For a super-edge € = (A,B), a subset S C xa U xe is called 
(é, 7)-partial iffor every é-m.p. (a,b), the set S contains at most 297° j-relevant 
copies of the vertex a or b. 


Consider some pair_of vertices (a,b) with a € A and b € B, for some pair 
of super-vertices A € Vj and B € V. Let j Ee J, qe [L] be a pair of indices. 
Consider a subset S C xa(j) U xs(7), and a vertex v € Hz4(j,¢). Recall that 
the set x»(7) is split into the disjoint union of the subsets xP) and x (j) by 
a binomial random partition (see Section 3, step 2c). The set Hg4(j,q) is split 
into a disjoint union of the two subsets H 7 (7,q) and H (7,q), and these two 
subsets have the same size M/2. Let 8; = SN xP), 5S. =SN x? (j). 

Let sı = [Si], s2 = |S2l, w = |x,’  G)| and u = |S N xa(j)|. Note that sy, 
$2, and w are random variables, and u is a fixed value that does not depend on 
random coins. 


Lemma 1. For asuper-edge č = (A,B) € E, a pair ofvertices a € A, b € B, 
a pair ofindices į € J, q € |L], a subset S © xa(j) Uxe(j) and a vertex v € 


Mali, d), P(S AN v) > (1-1/2). Ega P(S] = s1, XPO) = w) . 


w 


Consider a subset S C xa (j)U xB (j). Let S = S(S) be the subset of vertices 
of AU B that have copies in the set S. Note that the set S decomposes into a 
disjoint collection of the subsets S(a) = cp(star(a)) N S, for different vertices 
a € S. For a yet more general subset S C xa U xB, we will use the notation 
S; = S;(S) for the subset of vertices of AU B that have j-relevant copies in the 
set S. 

We will use the notation x to denote an arbitrary vertex x € S, both for 
x € Á and z € B. Let PN,(v) denote the subset of copies of the vertex x that 
have a non-zero probability of being connected to the vertex v € Ma(j,q) by 
the random permutation step. In other words, let j(v) be the index 7 such that 
v E€ Ma(j,q). If v € Mae(j,9)\He,c(j,¢), then set PN,(v) = 0. Otherwise, let 


i,(x) be the index z such that v € H; ® (j q). Then set PN,(v) = x (4). 


Definition 5. Consider afixed super-edge č = (A, B), a pair of indices j € Î, 
q E€ [L], a subset S C xa U xB, and a vertex v € Me(j,q). The exact partitions 
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of the set Mg(j,q) into the unions of the subsets Hz a(j,q) and He»(j,¢q) for 
different €-m.p.s (a,b), the exact partitions of the sets Hz „(j,q) into the unions 
of the subsets HE? (5, q) and H se. (j q) for different vertices x € AUB, and the 


binomial partitions of the sets xz(j) into the unions of the subsets x? (7) and 
yP (j) for different vertices x € AUB are called safe with respect to the pair 
(S,v) iffor every vertex x € 5;(S), |S N PN,(v)| < 27-3. 


Lemma 2. Let S bean (é,7)-partial subset of xa U xr (see Definition 4) of 
size |S| < X lnn, and v € Me(j,q) be a vertex. The probability that the exact 
partitions of the set Ma(j,q), and the binomial partitions of the sets Xz(j) are 
safe with respect to the pair (S,v) is at least 1/nè. 


Corollary 1. With probability at least 1/n®, for every vertex x € S, |PNz(v)N 
S| < 27-8, 


4.2 Large Rounds and Their Effect 


In this section we consider the probability space that is determined by the ran- 
dom permutation and mixing steps (steps 2d - 2e) of the reduction, and, in 
particular, the partitions of the sets X+(j) are fixed for all the vertices z € AUB 
(and the index 7 € J is fixed as well). 


Definition 6. We say that a round R is j-large with respect to a super-edge 
é= (A,B) ifRs = RN (xa U xB) is of size greater or equal to c- 23+'Inn, 
where c = 2co + 2 (co is a universal constant mentioned in Section 3; it will 
be determined later). Otherwise, the round is called j-small with respect to the 
super-edge È. 

A schedule II is called short if its length is at most log? n/100. 


Consider a short schedule J. Let 7s(j) (respectively, Ma(j)) be the subsched- 
ule of M that contains only j-small (resp, j-large) rounds with respect to the 
super-edge ë. Note that a vertex v E€ Mg(j7) is informed by the schedule H if 
and only if either the subschedule 7(j) or the subschedule Me(j) informs v (or 
if both do). 

Let Ve(me(3)) = Urersg) EN (xa U XB) = Uren. ;) Re, and, analogously, 
Ve(He(9)) = Ure mi) Ra. 

Next, we show that the probability that the subschedule J7g(7) informs a 
significant number of vertices is negligibly small. . 

For a fixed super-edge ê = (A, B) € E, and a fixed index j € J, consider a 
subset M’ C Mz(j) such that for every index q € |L], |M’ N Mz(j,q)| < 1. Such 
subsets are particularly convenient for the analysis, since for a pair of distinct 
indices q1, q2 € [L], and a pair vertices vı € Me(j, q1), ve € Me(j, q2), the events 
(x C vı) and (x C ve) are independent. 


Lemma 3. For a fixed shortschedule ITI, with probability atleast 1 — TO 
there is no subset M’ C Mg(j) of size N = n that satisfies |Mz(j,q) OM'| <1 
for every index q € [|L], such that thesubschedule Ha(j) informs all the vertices 
of the subset M”. 
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4.3 Pivots 


We start with describing a combinatorial lemma from [ABLP91]. We will use 
this lemma in the rest of our analysis. 


Lemma 4. [ABLP9/] Let G = (Vi, V2, E) be a bipartite graph with |\Vi| = n. 
Let If = (Ri, Ro,..., Ri) be a collection of at most ae subsets of Vı. Then 
there exists a subset S of the vertices on the left-hand side of the graph (S © V1), 
and an index j € J, such that 


1. {S| < X -logn. 

2. Let II' = (R4, Rb,...) = INS. Then for every subset R' € IT’, |R’| > 23. 

3. Let fy be the number of subsets R! with cardinality 27+* < |R'| < 29+F+!, 
Then Y yso% < logn. 


Fix a super-edge ë = (A,B), and consider the subgraph of the graph G = 
(V, E) induced by the vertex set (xa U xB) U Me. Let H|e be the schedule 
IT restricted to the set x4 U xB. Apply Lemma 4 to this subgraph and the 
collection M| of subsets of the set x4 U xB, and let S and j be the subset 
and the index whose existence are guaranteed by the lemma. (Observe that 
Ixa Uxp| = n°? -|AU B] =n.) 


Definition 7. (1) The index 3 as above is called the pivot of the super-edge é 
with respect to the schedule IT. (2) A round R is small with respect to the super- 
edge č if it is 7-small with respect to super-edge €, and 9 is the pivot of č. (3) 
The schedule II is called €-proper if Vs(ma(j)) is an (€,j)-partial set, where the 
index j E€ J is the pivot of the super-edge € with respect to IT. 


4.4 Small Rounds 


Consider a schedule JT = (R1, R2,..., Ri), t < (log? n)/100, and a super-edge 
é = (A,B). Let Hle = (Rila, Rolz,..., Rila), Rele = Re A (va U xB), be the 
subschedule of IT restricted to the super-edge é (for simplicity of notation we 
assume that for every index £ € |t], the set Reļe is not empty; otherwise the 
number of rounds in the restricted schedule would be smaller than in the original 
one). 


Lemma 5. For a fixed super-edge č € É, anda fixed é-proper schedule IT, 
consider the L sets Mz(j,q), q E€ [L]. For each index q € [L], let Xq denote the 
indicator random variable of the event (na( j) INF Msz(j,q)). Let X = aa Ag 
Then P(X < 2n™® ~ n) > 1 — ezp(—NR(n®)). 


Lemma 6. For a fixed super-edge €, and a fixed short schedule IT, 
P(IT INF Me(j)) < ezp(~R(n®)),where j is the pivot of the super-edge è with 
respect to the schedule II. 


Definition 8. A schedule II is called proper if there exists a subset H CE that 
contains at least one half of all the super-edges, and such that the schedule IT is 
é-proper with respect to every super-edge č € H. Otherwise, the schedule IT is 
called non-proper. 
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Consider a NO-instance. Let IT denote the collection of all short proper 
schedules. 


Lemma 7. With probability at least 1 — exp(—2(n! log? n)), there is no ad- 
missible schedule H € I. 


4.5 Deriving the Results 


In this section we glue together all the pieces ofthe analysis and derive our main 
results. The details are omitted from this extended abstract. 


Theorem 2. Unless NP C BPTIME(n?(e8le”)) there exists a universal 
constant c such that no (deterministic or randomized) polynomial time algo- 
rithm may provide an additive (c- log? n)-approrimation guarantee for the Radio 
broadcast problem. 
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Abstract. For a prime p, max-2linp is the problem of satisfying as many 

equations as possible from a system of linear equations modulo p, where 
every equation contains two variables. Hastad shows that this problem 
is NP-hard to approximate within a ratio of 11/12 + e for p = 2, and 

Andersson, Engebretsen and Hastad show the same hardness of approx- 
imation ratio for p > 11, and somewhat weaker results (such as 69/70) 
for p = 3,5, 7. We prove that max-2linp is easiest to approximate when 
p = 2, implying for every prime p that max-2linp is NP-hard to approxi- 
mate within a ratio of 11/12+ e. For large p, we prove stronger hardness 
of approximation results. Namely, we show that there is some universal 

constant > 0 such that it is NP-hard to approximate max-2linp within 
a ratio better than 1/ př. We use our results so as to clarify some aspects 
of Khot’s unique games conjecture. Namely, we show that for every € > 0 
it is NP-hard to approximate the value of unique games within a ratio 
of €. 


1 Introduction 


Systems of linear equations can be solved in polynomial time using Gaussian 
elimination. However, max-lin, the problem of finding a solution that maximizes 
the number of satisfied equations (in a system that is nonsatisfiable) is NP-hard. 
Here we consider the approximation ratios achievable for max-lin, where for a 
given algorithm this ratio is taken to be the number of equations satisfied by 
the algorithm divided by the number of equations satisfied by the best solution. 
In the following discussion we parameterize max-lin by two parameters, k and 
p, and denote the resulting problem as max-klinp. Here, k is the number of 
variables per equation, and p is a prime, signifying that all computations are 
performed modulo p. 

For every system of linear equations modulo p, a simple greedy algorithm is 
guaranteed to satisfy at least a 1/p fraction of the equations. Hence for every k 
and p, max-klinp can be approximated within a ratio of 1/p. A celebrated result 
of Hastad [8] shows thatfor k = 3, this is essentially best possible. That is, for 
every prime p and for every € > 0, max-3linp is NP-hard to approximate within 
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a ratio of 1/p+e. In particular, max-3lin2 is NP-hard to approximate within a 
ratio better than 1/2 + e€. 

The situation for the case k = 2 is different. Here there are approximation 
algorithms with approximation ratios significantly better than 1/p. Goemans 
and Williamson [7] gave an approximation algorithm based on semidefinite pro- 
gramming that approximates max-2lin2 within a ratio of roughly 0.87856. This 
result was extended in [2,9] to show that for every prime p there is some € that 
depends on p such that max-2linp can be approximated within a ratio better 
than (1 + €)/p. The value ofe in these proofs converges to 0 as p grows. 

In terms of hardness results, Hastad [8] shows that max-2]lin2 is NP-hard to 
approximate within a ratio of 11/12 + e. In [2] it is shown that it is NP-hard to 
approximate max-2linp within a ratio of H, 55, 33 (+€, in all cases) for p = 3,5,7 
respectively, and within a ratio of 11/12 + e for all p > 11. This state of affair 
raises the following questions. 


1. The approximation ratios of known algorithms for max-2linp deteriorate and 
tend to 0 as p grows. The hardness results to not exclude approximation 
ratios as good as 11/12 for every p. Does, indeed, the best possible approx- 
imation ratio for max-2linp tend to 0 as p grows? This question was asked 
explicitly in [2]. In this paper we give a positive answer to this question. 

2. The hardness of approximation ratios for small values of p (p = 3,5,7) are 
not as strong as those for p = 2. Is it indeed easier to approximate max- 
2linp when p = 3,5, 7 than it is when p = 2? In this paper we give a negative 
answer to this question. 


In the rest of the introduction we discuss our results in more detail. In partic- 


ular, we also show how our results are connected to the unique games conjecture 
of Khot [9]. 


1.1 Small Values of p 


A reduction between two optimization problems is approximation preserving if 
there is some constant c > 0 such that the optimum value of the target prob- 
lem is exactly c times the optimum value of the source problem. A polynomial 
time approximation preserving reduction from problem A to problem B can be 
used to transform any polynomial time approximation algorithm for problem B 
into a polynomial time approximation algorithm for problem A, with the same 
approximation ratio. 


Theorem 1. For every prime p = 3, there is apolynomial-time approximation- 
preserving reduction from max-2lin2 to max-2linp. 


Theorem 1 is proved by using local gadgets. Every equation modulo 2 is 
replaced by a collection of equations modulo p according to some fixed rule, in- 
dependently of all other equations. However, the analysis of why this reduction 
is approximation preserving is global (considering all equations simultaneously) 
rather than local (analyzing every gadget application separately). The global 
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nature of the analysis of our gadgets may be the reason why despite their sim- 
plicity, they have not been discovered earlier. In particular, Trevisan, Sorkin, 
Sudan and Williamson [12] present a systematic way (based on linear program- 
ming) for finding the optimal local gadgets in reductions, but their approach (as 
presented in [12]) is limited to gadgets whose analyses are local. 

Using the fact that it is NP-hard to approximate max-2lin2 within a ratio 
strictly of 11/12 + e, Theorem 1 implies: 


Corollary 1. For every e >0 and prime p, it is NP-hard to approximate max- 
2linp within a ratio of 11/12 + e. 


The following question remains open. Let p and q be two arbitrary primes 
s.t. p > q. Observe that the discussion in the introduction implies that the 
approximation ratio for max-3ling is strictly better than that for max-3linp. Is 
the approximation ratio for max-2ling strictly better than that for max-2linp? 


1.2 Large Values of p 


When p is sufficiently large, the bounds implied by Theorem 1 can be greatly 
improved. 


Theorem 2. There is some 6 > 0 such that for every prime p, max-2linp is 
NP-hard to approximate within a ratio better than 1/p°. 


The proof of Theorem 2 is by a reduction from a problem that is sometimes 
called label cover [4], a notion recalled under different terminology in Theorem 3. 


1.3 The Unique Games Conjecture 


The unique games conjecture of Khot [9], if true, has far reaching consequences 
regarding approximation of optimization problems. In particular, Khot and 
Regev [10] show that it would imply that vertex cover is NP-hard to approximate 
within a ratio of 2 — e. We recall this conjecture. The notation and terminol- 
ogy that we use in our presentation were chosen so as to clarify the connection 
between unique games and the problem of max-2lin. 

We denote a (one round, two provers) game by G = G(X,Y,A,B,Q,P). 
There are two finite sets of variables, X and Y, and two finite sets of values, A 
for X, and B for Y. There is a support set Q C X x Y. In the most general form 
of games, there is a probability distribution m defined over Q. In the context 
of the current paper, we may take m to be uniform, and so as to simplify the 
presentation, we omit all references to m. For every pair of variables (x,y) € Q, 
there is a polynomial time computable predicate P,,y(x,y). (More explicitly, 
the pair (x,y) plays two roles here, both to specify which predicate is being 
evaluated, and as variables in the predicate itself. Equivalently, we may think 
of there being only one predicate P over four variables, the first two being the 
names of x and y, and the last two being the values of x and y. The predicate 
is polynomial time computable in the sense that given values for x and y, it 
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can be evaluated in time polynomial in |X| + Y|.) The goal of the game is to 
find two functions, Py : X — A and P, : Y — B, that satisfy the maximum 
number of given predicates. (That is, for every (x,y) € Q, we check whether 
P,y(Pi(z), Pe(y)) evaluates to true.) The value of a game G, denoted by w(G), 
is the maximum over the choices of P} and P} of the number of predicates 
satisfied, divided by the normalizing factor |Q| (which is the total number of 
predicates). Hence 0 < w(G) <1. 

A game is said to have oneway uniqueness if for every (x,y) € Q and every a € 
A, there is exactly one b € B such that P;,,(a, 6) holds. The following theorem 
is well known and has been used extensively in previous work on hardness of 
approximation. 


Theorem 3. For every € > 0 there are sets A and B with |A\+|B| < (1/e)°™, 
such that it is NP-hard to distinguish between oneway unique games G with 
w(G) = 1 and oneway unique games G with w(G) < e. 


Proof. For completeness, we sketch the proof (which can be regarded as folk- 
lore). The starting point is the fact that it is NP-hard to distinguish between 
satisfiable 3CNF formulas and those that are at most 6-satisfiable, for some 
ô < 1 [5]. From this one can design a oneway unique game G in which X is the 
set of clauses, Y is the set of variables, and Q contains all clause-variable pairs 
where the variable appears in the particular clause. Every clause has 7 satisfying 
assignments. Encode these assignments as numbers in {1,.., 7} (where 7 encodes 
the ith satisfying assignment in lexicographic order). The set A is satisfying as- 
signments to clauses encoded as explained above, and the set B is assignments 
to single variables. Predicate P,,(a,b) (where y is a variable in clause x) holds 
if the triple assignment a satisfies the clause x, and a and b are consistent in 
their assignment to y. w(G) = 1 if the original 3CNF formula is satisfiable, and 
w(G) = (2+ 4)/3 if the original formula was only 6-satisfiable. Thereafter, the 
parallel repetition theorem of Raz [11] gives the desired result. | 


To avoid the possibility of misunderstanding, let us point out that in The- 
orem 3 (as well as elsewhere when we discuss games) we view the sets A and 
B as fixed for a whole class of games, whereas other parameters of the games 
(X, Y,Q,P) change within the class of games. Hence the notion of NP-hardness 
relates to algorithms whose running time is polynomial in |X| + [Y]. 

A unique game has the oneway uniqueness property in both directions, 
namely, also for every b € B there is exactly one a € A such that P, „(a,b) 
holds. The unique games conjecture of Khot [9] says that a weaker form of The- 
orem 3 holds also for unique games. 


Conjecture 1. (The unique games conjecture of Khot) For every 0 < e < 1/2 and 
every 0 < 6 < 1/2, there are sets A and B such that it is NP-hard to distinguish 
between unique games G with w(G) > 1—e and unique games G with w(G) < 6. 

Equivalently, for every € > 0 and every 0 < 7 < 1, there are sets A and B 
such that it is NP-hard to distinguish between unique games G with w(G) > 7 
and unique games G with w(G) <e€-7. 
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The condition 7 < 1 cannot be strengthened to 7 = 1 because there is a 
polynomial time algorithm that decides whether w(G) = 1 for unique games. 

A weaker version of the unique games conjecture simply asks whether for 
every €, it is NP-hard to approximate w(G) for unique games within a ratio 
better than e. Technically, this weaker version replaces the condition “for every 
€ > 0 and every 0 < ny < 1” in Conjecture 1 by the weaker condition “for 
every € > 0 there is some 0 < 7 < 1 such that”. Khot observed that even this 
weaker form of the unique games conjecture was not known to be true, and asked 
whether it is true. We give a positive answer to this question. Our approach is 
explained in the following. 

Motivated by the unique games conjecture, we prove a stronger version of 
Theorem 2. Call a system of equations modulo p proper if it has the following 
properties: 


1. The set of variables can be partitioned into two disjoint sets, X and Y. 

2. Every linear equation contains exactly one variable from X and exactly one 
variable from Y (with nonzero coefficients! ). 

3. For every pair of variables x € X and y € Y, there is at most one equation 
containing both x and y. 


Proper max-2linp is the problem of finding an assignment that maximizes 
the number of satisfied equations in a proper instance of 2linp. With some extra 
work, the proof of Theorem 2 can be extended to show that: 


Theorem 4. There is some 6 > 0 such that for every prime p, proper max-2linp 
is NP-hard to approximate within a ratio better than 1/p°. 


As a corollary, we have that unique games are hard to approximate within 
any € > 0. 


Corollary 2. For every e > 0, there are some constant 0 < n < 1, prime p, 
and sets A, B with |A| = |B| = p, such that it is NP-hard to distinguish between 
unique games G with w(G) > 7 and unique games G with w(G) < €- 7. 


Proof. A proper 2linp system S of equations is a unique game G. The sets X, Y 
of variables for S serve as the sets X, Y of variables for G. The set of values 
A, B for G corresponds to the set {0,1,...,p— 1}. The support Q corresponds 
to all pairs of variables (x,y) that appear in some equation in S. As each pair 
(x,y) € Q appears in exactly one equation in S (because S is proper), this 
equation serves as the predicate Py, for G. The uniqueness property follows 
from the fact that modulo p, every equation of the form cx = d (with nonzero 
coefficient c and arbitrary d) has a unique solution, and similarly for cy = d. 
Having established the fact that a proper 2linp system is a unique game, it 
remains to choose the parameters of the system. Choose p large enough such that 
1/p in Theorem 4 is smaller than e. Then for proper max-2linp there is some 
constant 0 < 7 < 1 such that it is NP-hard to distinguish between instances 
in which an 7-fraction of the equations are satisfied, and instances in which at 
most an €- 7-fraction of the equations are satisfied. As desired. E 
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(Note: an explicit value for 7 can in principle be derived from the proof of 
Theorem 4, though at the moment there does not seem to be much interest in 
doing so). 


1.4 Equations over the Rationals 


In all our reductions, if one treats the system of equations as a system over the 
rationals (and then we call the underlying problem max 2lin, omitting the p) 
or over the integers, rather than modulo p, the same hardness of approximation 
result follow. As the prime p increases, the approximation ratios proved hard in 
Theorem 2 tend to 0. One obtains: 


Theorem 5. For every e > 0, it is NP-hard to approximate max-2lin within a 
factor of €. 


Using our techniques, one can also prove hardness of approximation results 
for max-2lin within ratios that tend to O as the size of the input instance increases 
(e.g., of the form e = 270087)" for y < 1, where n is the number of equations in 
the system). However, these results are proved under the assumption that NP 
does not have quasi-polynomial time algorithms. It is an open question whether 
such results can be proved under the assumption that P 4 NP. 

Theorem 5 and the discussion that follows should not be confused with hard- 
ness results appearing in [1,3], as there the number of variables per equation is 
not limited to 2. Additional discussion on the difference between 2 versus many 
variables per constraint can be found in [6]. 


2 Proofs 


We present here the proofs of our main theorems (except for those whose proof 
was already sketched in the introduction). 


2.1 Small Values of p 
We restate Theorem 1 in more detail, and provide the proof. 


Theorem 6. For prime p > 3 There is a polynomial time reduction transform- 
ing an instance E of max 2lin2 to an instance F of max 2linp s.t. one can satisfy 
k equations in E iff one can satisfy p- k equations in F. For p = 3, there is a 
polynomial time reduction transforming an instance E of max 2lin2 to an in- 
stance F of max 2lin3 s.t. one can satisfy k equations in E iff one can satisfy 
2k equations in F. 


Proof. Every equation in E is of the form x + y =o 0 or z + y =2 1. Consider 
first the case in which p > 3. Transform an equation of the first form to the set 
of 2(p — 1) equations 

L =p ly 


On Systems of Linear Equations with Two Variables per Equation 123 


for ¿i = 1,...,p— 1 and 
x— 1 =, ify —1) 


for 2 = 1,...,p — 1. Transform an equation of the second form to the equations 
L-1=py ry 


forz=1,...,p—1 and 
£ =p ify — 1) 

fori = 1,...,p — 1. Call such a group of equations a block. Assignments of 
0/1 values to x and y that satisfy an original equation will satisfy exactly p 
equations in the corresponding block. Assignments of 0/1 values which do not 
satisfy the original equation do not satisfy any equation in the corresponding 
block. Non-0/1 assignments to either x or y satisfy at most two equations in the 
corresponding block. 

This proves that if one can satisfy k equations in E, one can satisfy p-k 
equations in F. To see the converse look at a satisfying assignment o of F 
satisfying p- k equations. If all the variables receive 0/1 values the claim follows 
immediately (Give to all the variables in E the values given to them by ø ). If not, 
call every variable not receiving a value from {0,1} misbehaved, and assign to all 
misbehaved variables independently at random 0 w.p. half and 1 w.p. half. Every 
block of equations containing a variable not set to values in {0,1} will have p of 
its equations satisfied with probability 1/2 and zero of its equations satisfied with 
probability 1/2. Hence the expected number of satisfied equations in such a block 
is p/2. We conclude that there is a 0/1 assignment to the misbehaved variables 
such that if we have l blocks containing misbehaved variables the number of 
satisfied equations in those blocks will be at least pl/2. ø satisfied at most 2/ 
equations in those l blocks. As p/2 > 2 (recall we assumed that p > 3) we get 
a 0/1 assignment satisfying more equations than ø. In particular it satisfies at 
least p - k equations and we are done as we have seen that a 0/1 assignment 
satisfying p- k equations of F implies we can satisfy k equations in £. 

In case p = 3 transform an equation of the type x+y =2 0 to the 3 equations 


T — Y =3 0 
xr — 1 =3 2(y — 1) 
zt =3 2y 


An equation of the type x + y =3 1 will be transformed to the 3 equations 


xT =3 2(1 — y) 
x-l=sy 
Lt=3l-y 


One can check that assigning the variables 0/1 values which satisfy the original 
equation will satisfy two out of the three equations, assigning the variables 0/1 
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values which do not satisfy the original equation will satisfy zero out of the three 
equations and giving to at least one of the variables value 2 will satisfy at most 
one equation out of the 3. An analogue argument to the one given in the last 
paragraph shows that if we have an assignment satisfying 2k equations in the 
new system we have such an assignment with 0/1 values (in our random process 
every block containing a misbehaved variable will have an expected number of 
(2 +0)/2 = 1 satisfied equations). The rest of the arguments are as in the above 
paragraphs. u 


2.2 Large Values of p 
Here we prove Theorem 2. 


Proof. The proof is by reduction from Theorem 3, and the reader is advised to 
recall the relevant notation for games. For simplicity in our presentation, rather 
than fix p in our proof and then figure out the relevant parameters for the game 
G, we shall fix parameters for the game G, and based on them choose p. It is 
not difficult to exchange the order of these choices, and we omit the details. 

Consider an arbitrary game G with oneway uniqueness and |A| > |B| (as 
is indeed the case in the games appearing in the proof of Theorem 3), and let 
p = O(|A|/e),for € as in Theorem 3. Let o be an arbitrary one to one mapping 
from A to Zp and 7 be an arbitrary one to one mapping from B to Zp. Thus 
from now on we look at elements of A and B as elements of Zp 

For every pair (x,y) € Q, replace the predicate Pp y by a blockof |A|(p — 1) 
equations as follows. For every a € A the block contains p—1 equations gz — a = 
i(y — b), where 1 <i < p—1, and b € B is the unique value s.t. P, (a,b) = 1. 
Looking at a group of p — 1 equations at a specific a, we see that an assignment 
that zeroes (x — a) and (y — b) will satisfy all p — 1 equations whereas any other 
assignment will satisfy at most one equation. 

For any assignment assigning c to x and d to y where (x,y) € Q we have that 
if Pz,y(o—!(c),7~1(d)) = 1, at least p — 1 equations will be satisfied out of the 
block corresponding to (x,y). If on the other hand, P,.4(a~+(c),7~1(d)) = 0, at 
most |A| equations would be satisfied in the (x,y) block. We remark that in any 
event we cannot satisfy more that p — 1+ |A| — 1 equations per block. 

Since we have a block of equations for every pair (x,y) E€ Q, we get that if 
w(G) = 1 then we could satisfy at least (p — 1)|Q| equations. 

Assume now that 7 is an assignment that satisfies more than ((1 — e)| A| + 
e(p—1+|A}]—1))|Q]| equations, where € is as in Theorem 3. Since 7 cannot satisfy 
more than p— 1+ |A| — 1 equations per block, it must hold that the fraction of 
blocks inwhich 7 satisfies more than |A| equations is strictly larger than e. Hence 
by the discussion above it follows that for G, the assignment P, (x) = o~1((2)) 
for all x € X and Pa(y) = 7~!(a(y)) for all y € Y shows that w(G) > e. Thus 
we get a hardness of (recall that |A| < p) 


(1) /Al + e(p—14]4]-1)1Q] 14l 
(p — 1)|Q| ~ pol 
As p = O(| Aj /€) and |A| = (1/e)°™, the proof follows. E 


+ 2e 
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2.3 Proper Max-21inp 


The proof of Theorem 2 gives a system of equations arranged in blocks. To every 
pair of variables (x, y) € Q corresponds one block of exactly |A|(p— 1) equations. 
In what follows, the parameters q and m will be used to denote q = |A|(p — 1) 
and m = 40gp*. In our proof that proper max-2linp is hard to approximate, we 
shall make use of a notion that we call balanced coloring. 


Definition 1. Let c be an edge coloring with q colors of the complete bipartite 
graph Km,m. Call a vertex-induced subgraph G = (V, E) big if |E| > m?/p*. We 
say c is balanced ifthe following two conditions hold: 


1. Every color is assigned to at least m?/2q edges. 
2. For every big induced subgraph G = (V, E), every color is assigned to at most 
3(|E|/q) edges in E. 


Lemma 1. Balanced colorings as defined above, exist. 


Proof. We give an existence proof using the probabilistic method. Color all the 
edges of Km,m with q colors by assigning independently at random to each edge 
a color between 1 and q (each color is chosen with probability 5). Let G = (V, E) 
a fixed big induced subgraph of Km,m. The probability that a given color will 
occur more than 3|F|/q times or less than $|E|/q in this induced subgraph is 
by the Chernoff Bound at most 


m2 
9 ; (1/e)®38:4lEl/4 < 9 , (1/e) za 


The number of induced subgraphs of Km,m is bounded by 2°™. Taking the 
union bound on all colors and all induced subgraphs we get that the probability 
some big subgraph will not have the desired property is bounded by 


As we took m to equal 40p%q the exponent of the above expression is positive. 
Hence, with positive probability we get a balanced coloring and the result follows. 
E 


In our proofs we will need balanced colorings when the parameters m and 
q can be regarded as constants (even though they are “large” constants), and 
hence a balanced coloring can be found in constant time by exhaustive search. 

Let E be an instance of max 2linp with the block structure as described 
above. That is, every block has q equations and all the equations in a block 
contain the same two variables (and every equation contains an X variable and 
a Y variable). For this reason, this 2linp system is not proper, in the sense defined 
in Section 1.3. We reduce the improper (by improper we mean an instance that 
is not proper) instance of 2linp (that we call E) to a proper instance of 2linp 
(that we call E’). 

The set of variables for Æ’ willbe m times as large as that of E. Every variable 
x E€ X for E will be “replaced” by m variables 21,..., £m in E’. Likewise, y € Y 
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will be replaced by yj,...,%m- AS to the equations, every block of q equations 
in E is replaced by a block of m? equations in E’. This is done as follows. Let x 
and y be the two variables occurring in a block of E. Assign to every equation 
in the the block a different color from 1, 2.....,q (in arbitrary order). Color the 
edges of Km,m witha balanced coloring with q colors. If in this coloring the color 
of the edge between 7 and 7 is l, put x; and y; in equation l. We remark that this 
coloring is used simultaneously in all the blocks of equations of |£’| and that the 
number of blocks in Æ” is |Q], the same as the number of blocks in E. It is not 
hard to see that Æ’ is indeed proper. 

Denote by X’ the set of all the x variables in Æ’ and by Y’, all y variables 
in F’. Say a set R= S xT C X’ xY” is a rectangle if both S and T are subsets 
of the variables in the same block of E’. A rectangle is big if it contains at least 
a 1/p? fraction of the equations in a block. 

Let r be an arbitrary assignment to the variables of E’. For any block of 
E’, let x1,...,2m;,Y1;-++;Ym be its variables. Assignment 7 partitions both 
T1,- --; Zm and y1,...,Ym to at most p nonempty sets Sk, Tg where x; belongs to 
the S; if7(a;) is the jth value from the field and similarly for T; with the y vari- 
ables. By the properties of balanced coloring, in every big rectangle R = S; x Thn 
any color in R appears t|R|/q times where 1/2 < t < 3/2. (|R| = |S;| x Thl). 

The proof of Theorem 4 now follows from the combination of Theorem 2 and 
the following lemma. 


Lemma 2. Let n denote the optimum’ of the improper 2linp system E with 
block structure as described above. Then 7, the optimum of the proper system 
E' resulting from the reduction described above, satisfies n/2 < n! < 3n. 


Proof. Let o be an assignment that satisfies ņnqg|Q| equations in E. Define an 
assignment 7 to the variables of E’ as follows: for each tuple of m variables 
Z1,.+-,;2m Corresponding to a variable z of the original system put 7(z1) = 
T(z2) =... =T(Zm) = a(z}. By property 1 of balanced coloring we have that 7 
satisfies at least 57m?|Q| equations in the new system. Hence 7’ > 7/2. 

We now prove that 7’ < 3n, or rather, that n > 7/3. Let 7 be an optimal 
assignment to the variables of the new system, satisfying 7/m?|Q| equations. 
Index the blocks with numbers between 1 and |Q]. Let k be an arbitrary block 
of equations in E’, and let x and y be the variables in the corresponding block of 
E. As explained above, the assignment 7 partitions the block k into rectangles 
Sik X Tjk (0 < i,j < p-— 1), where x variables are in S;, if they are given the 
value 2 by the assignment and similarly for y variables. For any big rectangle 
define ci j,k to be the fraction of equations of Sik x T}, satisfied by 7. The 
number of equations in rectangles that are not big is bounded by p? - as -m?, 


Thus, the number of equations satisfied by tin E’ is at most 


IQ! 
miQ/p+ >, >, (Sxl Tieles,s,) (1) 


k=1 Si k XT}, k ts big 


' By optimum we mean the maximum fraction of equations that can be satisfied by 
an assignment. 
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Construct an assignment g to the variables of E as follows: choose k and ł 
uniformly at random from [1,..m]. For every x € X, let a(x) = T(£k) (where £k 
is the kth variable replacing z), and for every y € Y, let o(y) = T(y). Let Z be 
the following random variable: Z counts the number of equations of E satisfied 
by a. Its expectation satisfies: 


IQI os 
noss D A Be a) (2) 


m 
k=1 Si kXT}j,k is big 


The above inequality follows from property 2 of balanced colorings. Namely, 
in a big rectangle R no color appears on a fraction of more than 3 of the 
edges (equations). In a block of E, the same color appears on exactly 1/q of the 
equations. Hence in E the fraction of satisfied equations shrinks by a factor no 
worse than 2 compared to R. 

There must be some assignment to E that achieves at least the expectation of 
Z. Comparing equations (1) and (2), this implies that 7 > 2(n! —1/p). Moreover, 
7 > 1/p, as in every system of equations modulo p one can satisfy a 1/p fraction 
of the equations. Hence 7 > 7/3, as desired. i 
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Abstract. Utility functions satisfying gross substitutability have been 
studied extensively in the economics literature [1,11,12] and recently, the 
importance of this property has been recognized in the design of combi- 
natorial polynomial time market equilibrium algorithms [8]. This natu- 
rally raises the following question: is it possible to design a combinato- 
rial polynomial time algorithm for this general class of utility functions? 
We partially answer this question by giving an algorithm for separable, 
differentiable, concave utility functions satisfying gross substitutes. Our 
algorithm uses the auction based approach of [10]. 

We also outline an extension of our method to the Walrasian model. 


1 Introduction 


The recent papers of Papadimitriou [16] and Deng, Papadimitriou and Safra 
[5], which raised the issue of efficient computability of market equilibria, have 
resulted in considerable activity on this topic. Some of the algorithms are based 
on solving nonlinear convex programs [14,18], following the classical approach 
of Eisenberg and Gale [9], while others are combinatorial [6,10,13,7,8,17]. The 
latter algorithms are dominated by two main techniques: primal-dual-type algo- 
rithms which were initiated by Devanur, Papadimitriou, Saberi and Vazirani [6], 
and auction-based algorithms introduced by Garg and Kapoor [10]. 

Both these papers deal with linear utility functions. Moreover, both start 
with very low prices and monotonically raise them until equilibrium prices are 
reached. The usefulness of such a scheme was clarified in [8]: prices play the 
role of dual variables in primal-dual algorithms, and almost all such algorithms 
known in the areas of exact and approximation algorithms work by greedily rais- 
ing dual variables - arranging anything more sophisticated is extremely difficult. 
Furthermore, linear utility functions satisfy gross substitutability, 1.e., increas- 
ing the price of one good cannot decrease the demand for another. Hence, for 
such utility functions, monotonically raising prices suffices. In contrast concave, 
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and even piecewise-linear and concave, utility functions do not satisfy gross sub- 
stitutability, and designing market equilibrium algorithms for them remains an 
outstanding open problem. 

Linear utility functions have been generalized to spending constraint util- 
ity functions, which also satisfy gross substitutability, in [17]. Building on the 
algorithm of [6], a primal-dual-type polynomial time algorithm for spending 
constraint step utility functions has been obtained in [17]. The importance 
of gross substitutability has been recognized in the economics literature, and 
there have been attempts at designing algorithms for this general class of util- 
ity functions [1,11]; however, to our knowledge no such polynomial time al- 
gorithms are known. In this paper, we use the auction-based approach of [10] 
to give polynomial time approximation algorithms for computing equilibria for 
the general class of additively separable, differentiable, concave utility func- 
tions satisfying gross substitutability. Our algorithm achieves a complexity of 
O((E/e) log(1/e) log((evumaz)/(€minUmin)) log m) where E is the number of non- 
zero utilities u;;, vi; (£:j}) is the slope of u;i; w.r.t. zi; and e; are the endowments 
(in terms of money). In the complexity description, emin = min; €i, € = J ;] €i 
Umaz/Umin is the ratio of the largest slope to the least slope (we assume that 
this ratio is bounded), v = it v1;(a~) and e is the tolerance parameter to 
which equilibrium prices are computed. This algorithm is faster by a factor of 
O(n) as compared to the auction algorithm for the Arrow-Debreu model [10] and 
also extends the class of utility functions for which approximate market clearing 
can be achieved. 

Interestingly, while the algorithm’s framework remains as simple as in [10], 
the proof of correctness and convergence are complicated by the general nature 
of the utility functions. We are able to, in this paper, resolve the convergence to 
the equilibrium prices via a monotone change in prices for separable increasing 
functions satisfying gross-substitution and concavity. The problem of comput- 
ing equilibrium in the more general class of (non-separable) increasing concave 
functions satisfying gross substitutability is a challenging one. 

In Section 2 we define the market model and provide a characterization of 
gross substitutable functions. In Section 3 we outline our algorithm and prove 
correctness and the complexity bounds. Finally, we outline (Section 4) an exten- 
sion to the Arrow-Debreu model. 


2 Market Model 


Consider a market consisting of a set B of n buyers and a set A of m divisible 
goods. Buyer 2 has, initially, an amount of money equal to e;. The amount of 
good j available in the marketis aj. We will assume that the utilities are additive 
and separable. Buyer i has a utility function, U;(X;) = Ljus (xij) where £i; is 
the amount of allocation of good j to buyer i, Xi = (£il, Lig... Zim) represents 
the current allocation vector of goods to buyer i and u;;(x£;;) : Ry — Ry is the 
utility function of item 7 to buyer i. We assume that u;i; is non-negative, strictly 
increasing, differentiable, and concave in the range [0,a,;]. Let vi; represent the 
first derivative of U; w.r.t. xi; (which is well defined). Assume that the buyers 
have no utility for money, however they use their money to purchase the goods. 
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Given prices P = {p1, p2,...,Pm} of these m goods, a buyer uses its money to 
purchase goods that maximize its total utility subject to its budget constraint. 
Thus a buyer i will choose an allocation X; that solves the following buyer 
program B,(P): 


Maximize : DD Uiz (Ti; ) (1) 
1<jgm 
Subject to: ` Tijpj Š €i (2) 
1<j<m 
Vj - Tij > 0 (3) 


This defines a family of programs, each program parameterized by a fixed price. 
Since u;i; is concave for all ¿ and 7, the theory of duality (Kuhn-Tucker condi- 
tions) can be used to give the following necessary and sufficient conditions for 
optimality for a given price vector P: 


` TijPj = €i (4) 


1<j<m 
Vj aip; > Ui; (Zi) (5) 
Vj: £i; > 0 > agp; = Vij (Tij) (6) 
a; > 0,Yj : xiz > 0 (7) 


In the above conditions, œ; is the marginal utility per unit of money, also referred 
to as marginal bang per buck, for buyer i. If z;; > 0, then at z,; the marginal 
bang per buck for buyer i from good j ( u4;(2i;)/p;) is exactly œ;. Otherwise, 
vij (0)/p; < ai. We say that the pair (X, P), X = (X1, X2... Xn) forms a market 
equilibrium if (a) the vector X; € R% solves the problem B;(P) for user i and 
(b) there is neither a surplus or a deficiency of any good 1.e., 


Vi: >o ay =a; (8) 


l<i<n 


The prices P are called market clearing prices and the allocation X is called 
the equilibrium allocation at price P. 

The equations (8) and (4) imply that all the goods are sold and all the buyers 
have exhausted their budget. Equations (5) and (6) imply that (a) that every 
buyer has the same marginal utility per unit price on the goods it gets and (b) 
every good that a buyer is not allocated provides less marginal utility. 


2.1 Gross Substitutes 


Gross substitutes is a well-studied property that has useful economic interpreta- 
tions. Goods are said to be gross substitutes for a buyer iff increasing the price 
of a good does not decrease the buyer’s demand for other goods. Similarly, goods 
in an economy are said to be gross substitutes iff increasing the price of a good 
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does not decrease the total demand of other goods. Clearly, if the goods are gross 
substitute for every buyer, they are gross substitutes in the economy. 

We now give a formal definition of gross substitute in our model. Consider 
the buyer maximization problem B;(P). Let S;(P) C RY be the set of optimal 
solutions of the program B,(P). Consider another price vector P’ > P. Goods 
are gross substitutes for buyer i if and only if for all X; € S;(P) there exists 
X; €e S;(P") such that p; =p} ays Tij. 

Note that u;; is continuous, concave and differentiable for all ¿ and j and 
vij(£) = uj (x). Since ui; isconcave, vi; is a non-increasing function. The fol- 
lowing result characterizes the class of separable concave gross substitute utility 
functions. 


Lemma 1. Goods are gross substitutes for buyer if and only if for all j, yvi; (y) 
is a non-decreasing function of the scalar y. 


Proof. To prove the converse part (which is critical to the algorithm in Section 3), 

assume that there are scalars y and y’ such that y’ < y and y'vij(y') > yvij(y). 

Choose a price P and an optimal solution X; of B;(P) such that zi; = y. Let 

a; be the optimal dual solution of B;(P). Construct a corresponding P’ and X; 

FA that Tip = Zik, Pp = Pr forall k #9, xi, = y’ and p} = pyri (£i) /Vij (zij). 
ow, 


Li 5;Py = LijPjVij (Xjz)/ Vig (Tey) 
> p53 ig Vig (Liz) /Vig (Tiz) 
= Lip; 


So, the solution Xj; satisfies (11) and (10) for price P’, but Dist rip; > 
iat £Tijp; = e;. Therefore, the optimal dual solution œ; of B;(P’) will sat- 
isfy a; > aj. Therefore, the optimal solution Xj’ of Bi(P) will have x, < £; 
for all k Æ j, such that zik > 0. Hence the goods will not be gross substitute for 
buyer 2. 

We next show that yvi;(y) is non-decreasing then the goods satisfy gross- 
substitutability. Consider an optimal solution X; € S;(P). The dual of the 
program B;(P) gives the following necessary and sufficient conditions for the 
optimality of X;. 


Š Tupi = 6; (9) 
j=l 


Vj : zy > 0S 05 (ty) = aP} (10) 
Vj: apj È Vig (riz) (11) 
Qi > 0, Ziz > 0 


If xi; > 0, equation (10) gives a4;p; = TijVij/@i. Consider P’ > P. For 
this price vector we construct a feasible solution X; as follows: If p; = p; then 


Tij = Tij, Vi. Alternately, if vi; (0) < œip} then set zi; to zero, else choose 24, such 
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that v43(x;;) = asp;. By definition, the solution X; satisfies the complementary 
slackness conditions (10). Since P’ > P, X; also satisfies (11). Also since vij is 
a non-increasing function p} > pj > Tij < zij. Now, 


Lig Py = TijVig (Viz) /O% 
< Lig Vig (Lig) / 0% 
= TijPj 


The above equations give 


m m 

t / T , 
5 LP < 5 Lig Dj = ĉi 
j=1 


j=l 


Note that u;; is concave for all j. Therefore, there is an optimal solution Xj’ of 
the program B;(P’) such that Xj’ > Xj. (From the definition of Xj if pj = p 
then z;; = xij. Therefore p; = p} = xj; Z xij where X; is an optimal solution 
of B;(P) and X/’ is a corresponding optimal solution of B;(P’). 


3 An Auction Algorithm for Market Clearing 


We now present an ascending price algorithm for discovering the market clearing 
prices approximately. The algorithm starts with a low price and an initial allo- 
cation x; for all buyers 2, such that all the goods are completely allocated and 
optimal allocation of buyers dominate their current allocation. Now the prices of 
over-demanded items are raised slowly and the current allocation is recomputed, 
until no item is over-demanded. This approach has a similarity with the Hun- 
garian method of Kuhn [15] for the assignment problem. Unlike the Hungarian 
method which raises the price of all the goods in a minimal over-demanded set 
by a specific amount, our algorithm raises the price of one good at a time by a 
fixed multiplicative factor (1 +€), where € > 0 is a small quantity suitably chosen 
at the beginning of the algorithm. This algorithm has an auction interpretation, 
where traders outbid each other to acquire goods of their choice by submitting a 
bid that is a factor (1 +e) of the current winning bid. Prior to this, auction algo- 
rithms have been proposed for maximum weight matching in bipartite graphs, 
network flow problems and market clearing with linear utilities [4], [3], [2], [10]. 

We discuss the algorithm which is formally presented in Figure 1. In proce- 
dure initialize (Figure 1) all the items are allocated to the first buyer. Prices 
are initialized such that (a) the buyer’s money is exhausted and (b) the buyer’s 
allocation is optimal at the initial prices. An initial assignment of dual variables 
a; is also required. Instead of a; we maintain a,;, a separate dual variable for 
each item, for computational ease. Finally we will relate a; to ay;. 

It is easy to verify that the prices set in procedure initialize exhausts the 
budget of the first buyer. Since v1;(a;) is assumed to be strictly positive’, p; > 0 
for all 7. Therefore the initial value of œ;; is well-defined for all ¿ and 7. 


' This is not a necessary assumption. It is made for simplicity of the presentation. A 
weaker assumption would be that every item j has a buyer 2 such that vi;(a;) > 0. 
The initial allocation may still be found that satisfies the desired properties. 
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Also note that v1;(%4;)/pj; = a1; = a for all j. Hence the initial allocation 
maximizes the utility of the first buyer. 

The goods are allocated at two prices, p; and p;/(1+e). The allocation of good 
j to buyer i at price p; is represented by hij and the allocation at price p,;/(1+€) 
is represented by y;;. The total allocation of good 7 to buyer 7 is given by 
Tij = hi; +yij. Define the surplus of a buyer 7 as r; = Doj a; +yi;p;/(1+e)). 
Define the total surplus in the system as r = $; 4 Ti. 

The auction algorithm main (Figure 1) begins with a buyer 2 who has signifi- 
cant surplus (more than €e;) and tries to acquire items, with utility per unit price 
more than the current utility per unit price. It outbids other buyers by acquiring 
items at a higher prices. It raises the prices by a factor (1 + €) if needed. This 
process continues till total surplus in the economy becomes sufficiently small. 

The algorithm maintains the following invariants: (11) items are fully sold, 
(I2) buyers do not exceed their budget, (I3, 14) after completely exhausting its 
surplus a buyer’s utility is close to its optimal utility at the current prices, (I5) 
prices do not fall and (16) total surplus money in the economy does not increase. 
Figure 2 lists these invariants formally. 

It is easy to check that all the invariants (11 through I6) are satisfied after 
the initialization (1.e. after procedure initialize has been called). 

The allocation 2z;; is modified only in procedure outbid. However, the mod- 
ifications leave the sum } ,<,<,, Zi; unchanged. Therefore the invariant I1 is 
satisfied throughout the algorithm. 

For invariant 12, it is sufficient to show that r; > 0 for all 7. r; is reduced 
only in procedure outbid. In this procedure, the variable t2 is chosen such that 
ri does not become negative and hence I2 remains satisfied. 

For invariant I6, note that the only steps that change r are in procedure 
outbid. In these steps, r is reduced by et. Hence I6 is satisfied in the algorithm. 
The invariant I5 is trivially satisfied 

We now show that invariants I3 and I4 are satisfied by the algorithm. 


Lemma 2. During the approximate auction algorithm the invariants 13 and 14 
are always satisfied. 


Proof. The invariants are true initially for all the buyers. We first show invariant 
13. Note that when a,; is modified in algorithm main after calling outbid, the 
invariant is satisfied. Since p; never decreases, the invariant remains satisfied 
whenever p; changes. When z,; is reduced, v;; (xij) increases causing a potential 
violation of the invariant. In this case, the inner while loop of the algorithm will 
be executed. We argue that when the inner loop ends a;;p; = viz (zij) for all ¢, 7. 

To prove this, consider the time instant z when good j was acquired by 
buyer ż¿ at price p;. Let a be the quantity of good j acquired by buyer 2. Now, 
Qijpj = Vij(a). Assume that the amount of good j currently acquired by buyer i 
isb < a. Let the current price of j be p}. Choose c such that vi; (¢)/pi, = viz(a)/p;- 
It is always possible to do so since v4;(b) > aujp Z aijp; = vij (a). Since p} > pj, 
c < a. Now, cp; = cp;vij(c)/vij(a). From the assumption that goods are gross 
substitutes and using Lemma 1 we have cu;;(c) < avi;(a). Therefore cp, < apj. 


134 Rahul Garg, Sanjiv Kapoor, and Vijay Vazirani 


procedure initialize 
Yi, Yi : hg = 0 
Vi Æ 1, Y3 > Yay = 0 
Yi : Wig = Gij 
Vj : rg = (9; 2901; (aj))/e: 
Vj: py = viz (aj)/a 
Wa zÆ i- Nw; = vi; (0)/p5; Ti = Ei 
Vi # 1, Vj : cg = vij (wiz) / pj 
ry = 0 
end procedure initialize 


algorithm main 
initialize 
while Ji : T; > ee; 
while (r; >0) and (Jj : aijp; < vis (riz) 
if dak: yn; >0 then outbid(z, k, j, Qij) 
else raise_price(j) 
end while 
j = arg max; Qi 
if Jk : ye; >0 
outbid(i, k, 7, œx/(1+6)) 
Qij = Vij (£4) /p3 
else raise_price(j) 
end while 
end algorithm main 


procedure raise_price(j) 
VE2 Vij = hag; Rez =O; 
py = (1+ )p; 

end procedure raise.price 


procedure outbid(@i, k, j, a) 
ti = Yki 
to = ri/p; 
if (vjj(a;) > ap;) then tz =a; 
else t3 = minô : vij (ti +46) = ap; 
t = min(t;, t2, t3) 


he; == hej +t 
ri =T; — tpj 
Ykj =Ykj— t 


rk = Te + tpj/(1 +e) 
end procedure outbid 


Fig. 1. The auction algorithm 


Therefore the amount of money needed to be spent on j to ensure v;;(¢)p = aij 
is pje which is no more than pja; the amount of money spent on j before Tij 
was reduced. Hence, when the inner loop ends a,;p > vij (xij) for all ¿ and j. 
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Ih: V9: See =; 

2: V i: ae Tijpj © €i 

13: Yi i = 0s tij Pi > Vizli) 

I4: V 4,9: ty > 0> (1 + €) Vi; (xi) > AijPj 
I5:Y 3: pj does not fall 

16: r does not increase 


Fig. 2. The invariants in the auction algorithm 


The invariant I4 is satisfied after initialization. Whenever œ;; is changed in 
main vi; (zij) = a,;p;. Therefore, if zi; is reduced, I4 remains satisfied. x;; may 
be increased by a call to outbid in the inner loop. However, the parameter a;, 
in outbid ensures that the variable ta is chosen such that ayj;p; < Vij(£ij). 
Moreover, if x;; > 0 at the exit of the inner loop, then aj;p; = vij(xij). So, 
if p; is raised by a factor (1 + €), 14 will still be satisfied. If p; rises by more 
than the factor 1 + €, zı; will be set to zero and when 2;; is increased again 
QijPj = Vij (Liz). So, 14 will remain satisfied in the algorithm. 


—~— 


this stage the market-clearing conditions (4) and (6) are satisfied approximately 
and (5) and (8) are satisfied exactly. 


The algorithm ends when the surplus with each buyer is small, i.e. r; < €e;. At 


3.1 Convergence of the Algorithm 


In order to provide efficient convergence, bidding is organized in rounds. In each 
round every buyer (z) is picked once and reduces his surplus to 0, i.e. r; = 0. 
The following lemma proves a bound on the surplus reduction in each round. 


Lemma 3. Jn every round the total unspent money r = De r; decreases bya 
factor of (1 + 6). 


Proof. The value of r is decreased in procedure outbid by tep;. Buyer t bids 
until r; = 0. WLOG assume that these bids are of amounts t1,t2,...,tg on 
items 1,2,...,k& at prices pi, p2,...,px. Now we have: 


k 
Na(l +e)pi = ri 
i=] 


Reduction in r is given by 


k 
€ 
Ar = X tiepı = a Ti 
[=1 





Bidding by buyer z can only increase rą for another buyer k. Therefore the total 
reduction in r in one round is given by: 


n 
€ € 
Ar > —— 
ey a ras ite. 


i=1 
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The new value of unspent money r’ after every round is related to its old value 


u yh oT 
r aS: r tae 





Let emin = min; e; and e = $` ;_] €i. If r < €€min then no buyer has signifi- 
cant money left. Therefore the algorithm is guaranteed to terminate in k rounds 
where k = log(=— )/ log(1 + €). 

Let E be the number of non-zero utilities. We first bound the number of calls 
to procedures raise price and outbid. 


The price of any item is bounded by: 





CTE eaves 


Nes V1k (ak) aj E Qj 


At every step, the price is raised by a factor (1 + €), therefore the total number 
of calls to raise price is bounded by: 
ev 


— log( 
€ EminUmin 


where v = D V1; (ak), €Emin = min; e; and Vmin = mini; Vvik(ak) . We will 
assume that the ratio of v and Vmin is bounded. If not, Le. if min 1s zero then 
we can perturb the utility function u;;(x;;) by addition of the term ezi; such 
that the derivative is at least e. Furthermore, the derivative is bounded above 
by vik (ax). 

In a call to outbid, one of the following four events can occur: (a) yx; becomes 
zero for some k, (b) r; becomes zero, (c) œ;; reduces by a factor of (1 +€) and 
(d) vij (zij) reaches a,; in the inner while loop of algorithm main. 

Events of type (a) are charged to calls to procedure raise price and events 
of type (b) and (d) are charged to the current round of bidding. Events of type 
(c) are charged to a reduction in aj;;. 

Note that for every price rise of item j, the number of events of type (a) is 
bounded by the number of buyers having non-zero utilities on item 7. Thus the 
total number of type (a) events is bounded by E times the maximum possible 
number of price rises for any given item. 

It is easy to see that number of type (b) events is exactly equal to n in 
every round of bidding. At every type (c) event, œi; is reduced by a factor of 
(1+e).Since aj; = viz (x£i;j)/p; , its value varies from vi; (4;)/Pmaz tO vi; (0)/Pmin, 
where Pmin and Pmax are the minimum and maximum values of p;, respectively. 
In every round only one event of type (d) occurs for each buyer. 

During the algorithm evaluation of arg max; a, can be done efficiently by 
maintaining n heaps, one for each buyer. The heap of a buyer contains the 
goods with non-zero utilities sorted in decreasing order of œu. Updates requires 
O(log n) steps. 

This gives us the following time complexity of the algorithm: 


Theorem 1. The auction algorithm terminates in 
O((E/e) log((evumar)/(€€minUmin)) logn) steps. 


where Umax = Maxi; Vi; (0). 
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4 Extension to the Arrow-Debreu Model 


The algorithm for the Fisher model described in the previous section can be 
extended to the general Walrasian model, also called the Arrow-Debreu model, 
where there is no demarcation between buyers and sellers, and all agents come 
to the market with an initial endowment of goods. 

In order to extend to this case, we first note that under the assumption that 
goods satisfy gross substitutability, a version of Lemma 1 still holds, i.e., xu;; (x) 
is a non-increasing function with respect to buyer tif £ > dij. 

The algorithm is modified to compute surplus available with each of the 
traders based on the endowment of the trader. In each phase of the algorithm, 
a trader with surplus is picked. Firstly, items that do not satisfy the invariance 


aip; > vij(x) 


are acquired by being outbid at the current price, failing which the price of the 
chosen item is raised. Gross substitutability ensures that the invariance would 
be satisfied when the surplus is exhausted. Alternately, if every item satisfies 
the invariance w.r.t. the chosen trader with surplus, the trader bids on other 
items which may either be available or whose price may have to be raised. The 
bidding for items is performed in rounds, with each trader being considered at 
least once in a round and exhausting his surplus, to ensure fast convergence. A 
detailed analysis of the efficient convergence of this bidding process is similar to 
that provided in the paper of Garg and Kapoor[10] for the linear case. 


5 Conclusions 


It would be of considerable interest to extend the class of utility functions for 
which the market equilibrium problem is solvable via the auction method. In 
particular, can this approach be extended to non-separable functions? 
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Abstract. We consider a single source network design problem from 
a game-theoretic perspective. Gupta, Kumar and Roughgarden (Proc. 
35th Annual ACM STOC, pages 365-372, 2003) developed a simple 
method for single source rent-or-buy problem that also yields the best- 
known approximation ratio for the problem. We show how to use a vari- 
ant of this method to develop an approximately budget-balanced and 
group strategyproof cost-sharing method for the problem. 

The novelty of our approach stems from our obtaining the cost-sharing 
methods for the rent-or-buy problem by carefully combining cost-shares 
for the simpler problem Steiner tree problem; we feel that this idea may 
have wider implications. Our algorithm is conceptually simpler than the 
previous such cost-sharing method due to Pál and Tardos (Proc. 44th An- 
nual FOCS, pages 584-593, 2003), and has a much improved approxima- 
tion factor of 4.6 (over the previously known factor of 15). 


1 Introduction 


This paper studies the problem of giving good cost-sharing mechanisms for a 
single source network design problem. Imagine a general network design problem, 
where the participants (or agents) want to build a network connecting them to 
a common source (a server); however, they are autonomous and behave in a 
selfish (but non-malicious) fashion. Informally, a cost-sharing mechanism builds 
a network, and allocates the cost incurred among the agents, so that no group 
of agents is charged too much, thus precluding the possibility of their being 
unhappy and trying to secede from the system. 

The type of problem we consider is where we are given an undirected graph 
G = (V, E), and a set of demands D C V that want to connect to a common 
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source r. A cost-sharing method is an algorithm that builds a cheap network, 
and also specifies what portion of its cost is paid by which of the participants 
in the network; this is done in a manner that ensures that the cost paid by any 
subset of the participants is fair. 

Given an instance of a network design game specified by a set of demands D, a 
cost-sharing scheme for the problem is simply afunction €(D,z) with €(D,i) =0 
fort ¢ D. For a set D of demands we use OPT(D) to denote the minimum cost 
network serving the users in D, while Alg(D) denotes the cost, or expected cost, 
of the network computed by a given algorithm. There are three properties that 
we will be concerned with: 


1. (G-approximate budget-balance) For a set of demands D, we require 
that 


OPT(D) > Sep E(D, i) > Ale(D)/8 


for some given parameter > 1. Equivalently, by multiplying the shares 
up by 8, we could require that the cost-shares are at least the total cost 
of a solution found, but do not exceed 86 OPT(D). If 8 = 1, we call the 
cost-sharing budget-balanced. 

2. (fairness) For any A G D, 37,24 €(D,1) < OPT(A). Le., the cost paid by 
any subset of people should not exceed the optimal cost of connecting them 
alone and hence they have no incentive to secede. 

3. (cross-monotonicity) For any AC D, and demand iz € A we require that 
€(D,1) < €(A,i). Le., the cost-share of any demand should not go up due to 
other demands entering the system. This property is also known under the 
name of population-monotone. 


Cross-monotonicity is a key ingredient used in solving the following type of 
mechanism design problems: consider the network design problem with a set of 
demand nodes D, with each user (or demand) d having an associated utility 
ud. Since the users have limited utilities, the service provider has to now decide 
which subset of customers it must serve, in addition to designing the network 
and deciding how to share the cost between the served customers. A mechanism 
for solving this problem is called group strategyproof if no subset of users has 
an incentive to deviate from the protocol (e.g., by misreporting their utility) in 
the hope of improving the outcome for themselves (e.g., receiving the service 
at a cheaper cost). Moulin and Shenker [6] show that having a cross-monotone 
cost-sharing method for a problem naturally gives rise to a group strategyproof 
mechanism for the problem in the following way. We start with all the customers; 
if there is some customer whose cost share (computed w.r.t. the current set of 
customers) exceeds its utility, we drop it from the set, recompute cost shares 
and repeat. At the end, we are left with the desired set of customers and their 
cost-shares. 

As an example of cross-monotonicity, let us consider the MINIMUM SPANNING 
TREE game on the complete graph G = (V, E) with edge weights, given by a set 
of players D and arootr ¢ D; the objective is to find the cheapest tree MST(D) 
spanning D and r. It must be emphasized that this game does not allow the 
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use of Steiner vertices in MST(D), and r is not a player, and hence should have 
no cost-share. It is not difficult to verify that a budget-balanced and fair cost- 
sharing scheme for this game can be found thus: find an MST, root it at r, and 
set the cost-share for vertex 1 € D to be the cost of the edge from 2 to its parent. 
However, this scheme is not cross-monotone, and getting a cross-monotone cost- 
sharing scheme for the minimum spanning tree problem takes more work. Kent 
and Skorin-Kapov [5] and Jain and Vazirani [7] developed such budget-balanced 
and cross-monotone cost-sharing schemes for the spanning tree game using a 
directed branching game; let us denote this cost-sharing scheme by ys. Note 
that the values ig MsT Serve also as cross-monotone, 2-approximately budget- 
balanced cost-sharing for the corresponding Steiner tree game. 


The Single Source Rent-or-Buy Game. In this paper we will consider the Single 
Source Rent-or-Buy Network Design game; this combines features of Steiner 
tree and shortest paths. The game is defined as follows: we are given a complete 
undirected graph G = (V, E) with edge costs ce satisfying the triangle inequality, 
a special source (or roof) vertex r, and a parameter M > 1. There are also a set 
of users (also called players or demands) D C V, each of which is identified with 
a vertex of the graph. We assume that there is a unique player at each vertex. 
Our results can be easily extended to multiple players, each with a weight dj 
indicating its amount of demand; we leave the details for the final version of the 
paper. 

The objective is to connect each player j to the source r via some path P;, 
on which one unit of bandwidth has been allocated. What makes the game inter- 
esting is that aggregating paths is beneficial, in tune with the idea of economies 
of scale. Hence there are two different actions that can be performed by each 
edge: either the edge can be bought at cost Mce, but then an arbitrary amount 
of bandwidth can be sent on that edge (and hence an arbitrary number of paths 
P; can use it); or bandwidth on the edge can be rented at cost ce per unit band- 
width (and hence if the paths for some set S of players were using the edge, 
then the edge would cost ce x |S|). As usual, any edge can be used for any of 
the users. Our main theorem is the following: 


Theorem 1. There is a cross-monotone, fair cost-sharing method for the Single 
Source Rent-or-Buy network design game that is also G-budget-balanced, where 
b = 4.6. Furthermore, these cost-shares can be computed in deterministic poly- 
nomial time. 


This improves on the results of Pal and Tardos [9], who gave a 15-approximate 
cross-monotone cost-sharing scheme for the problem. They use a primal-dual 
algorithm to build the network and obtain cost-shares. 

We construct the claimed cost-shares based on expected costs in the approx- 
imation algorithm of Gupta et al. [4]. Loosely speaking, that algorithm works 
by randomly reducing the network design problem to computing Steiner trees 
over subsets of terminals; we show that if we use the function ig mst to allocate 
approximate cost-shares to each vertex in this resulting Steiner game, then the 
expected cost share of a vertex (taken over the random reduction) gives us the 
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cross-monotone cost-shares claimed in Therorem 1. However, since the variances 
of the random variables involved may be large in general, computing these ex- 
pected costs in polynomial time is non-trivial. To this end, we give an alternate 
analysis of the algorithm of [4]; this allows us to give a derandomization using a 
small sample space, which has some additional useful properties that help ensure 
cross-monotonicity. 

Note: Independently of our result, Leonardi and Schafer (personal communica- 
tion) have also suggested deriving cross-monotone cost-shares from SimpleCFL; 
however, lacking the derandomization of the algorithm, they are not able to 
compute these cost-shares in polynomial time. 


2 The Algorithm and the Cost-Sharing 


We will use an algorithm SimpleCFL suggested by Gupta et al. [4] for the equiv- 
alent problem of connected facility location (without facility costs). First, let us 
recall the algorithm from [4]. Here a > 0 is a constant that will be chosen later. 


S1. Mark each demand j independently with probability a/M, and let D’ be the 
set of marked demands. 

S2. Construct a minimum spanning tree T on F = D’ U {r}. (This is a 2- 
approximate Steiner tree on F.) The elements of F are called the open fa- 
cilities. 

S3. Assign each demand 7 € D to its closest demand i(j) in F. 


The algorithm suggests a simple and intuitive idea for the cost-shares: each 
player pays a cost proportional to the expected cost incurred by it on running 
the above algorithm. For a given a set of coin-tosses, the players in D’ will 
pay for buying the MST, where their shares are derived from a cross-monotone 
cost-sharing scheme msr for the MST problem given in [5,7]. All other players 
(players in D\ D’) will pay for their shortest paths to F. The cost-shares defined 
for a particular set of coin-tosses exactly cover the cost of the solution built. From 
[4] we know that this algorithm is a 6-approximation algorithm for some constant 
8. To get a -approximate cost-shares we divide the above defined shares by a 
ß. Formally: 

E(D, j) = 4 B[M êmsr(F, j) +40, F)], (2.1) 


where (j, S) is the length of a shortest-path from j to the closest vertex of the 
set S, and the expectation is over the coin tosses. (Note that the set F = D’U{r} 
depends on the coin tosses.) 
Outline of the Proof: There are two parts to proving the result: we first have 
to show that the properties of the cost-shares claimed in Theorem 1 are in- 
deed satisfied; i.e., they are G-budget-balanced, fair, and cross-monotone. These 
properties are proved in Section 3. 

The technically involved part of the proof involves showing that our cost- 
shares can be computed in (deterministic) polynomial time. A little thought will 
convince the reader that this is not obvious, even with the results of Gupta et 
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al. [4]. Indeed, we need to estimate the expectations in (2.1) for each of the 
players, but the random variables in (2.1) do not have small variance in general. 
Furthermore, it is unclear how to derandomize the proof of [4], since it relies on 
some severe conditioning. To take care of this problem, we give a proof of the 
performance guarantee of the SimpleCFL algorithm different from the one given 
by [4]; this is done in Section 4. Our new proof will yield a somewhat worse 
constant than that of [4], but allow us to derandomize the algorithm by marking 
the demands in Step (S1) in t-wise independent fashion for a constant t; this 
appears in Section 5. 

Using t-wise independent random choices in Step (S1), allows us to use a 
polynomial-sized sample space, letting us compute expectations such as those of 
(2.1) in (deterministic) polynomial time, by considering all the points in the sam- 
ple space. However, we need to make sure that the properties of the expectation 
proved in Section 3 (fair and cross-monotone) also hold for the expectation using 
t-wise independent random choices. Interestingly, the properties of a particular 
construction of t-wise independent random variables turns out to be crucial, as 
described in the proof of Theorem 4. 


3 Properties of the Cost-Sharing Scheme 


Recall that given an instance of the game specified by a set of demands D, a 
cost-sharing scheme for the game is simply afunction €(D,7) (with €(D,7) = 0 
for i ¢ D). We now need to show that the function defined in (2.1) is a cost- 
sharing scheme with the properties we care about, namely, approximate budget- 
balance, fairness, and cross-monotonicity. These three properties are, in fact, not 
independent. A cross-monotone and approximately budget balanced cost-sharing 
is always fair. 


Lemma 1. Jf a cost-sharing function € is approximately budget balanced and 
cross-monotone then it is also fair. 


Proof. Consider a subset A C D. By cross-monotonicity, we get }7,- 4 €(D,i) < 
ica &(A, i). The approximate budget balance property gives us 57 ,- 4 €(A,1) < 
OPT(A), and completes the proof of the lemma. 


We will need the following facts about the cost-sharing scheme Eysr : 


Theorem 2 ({5, 7]). There exists an efficiently computable cost-sharing scheme 
Eust for the Minimum Spanning Tree game, that is budget-balanced, fair and 
cross-monotone. 


Since the spanning tree connects D and r, we use the notation émsr(D U 
{r},z) and €ysr(D,i) interchangeably; however, we always ensure that 
Eust(D U {r},r) = 0. To prove approximate budget-balance, we need the fol- 
lowing result bounding the performance of the algorithm SimpleCFL; its proof 
appears in Sections 4 and 5. 
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Theorem 3. The expected cost of the algorithm SimpleCFL on a set of demands 
D is at most BOPT(D) (where B = 4.6), even if the demands are marked in a 
t-wise independent fashion in Step (S1), for a suitably large constant t. 


Note that this is a stronger form of the performance guarantee proved in [4] 
(albeit with a worse constant), since it requires only constant-wise independence 
of the random variables. Armed with this result, we are now in a position to prove 
the properties of the function é defined in (2.1), and thus to prove Theorem 1. 
The above theorems give us that € is -approximately budget-balanced: the 
details are deferred to the final version of the paper. 


Lemma 2. The function € is a B-approximately budget-balanced. 


Details of a Limited-Independence Marking: In order to prove that the ex- 
pectation using t-wise independent random marking is cross-monotone, we need 
a particular choice of random marking. Let F be a field of size > n, with |F| chosen 
large enough so that fa|F|/M]|/|F| is sufficiently close to a/M; for convenience, 
we will assume that this fraction equals a/M. (This approximation impacts our 
analysis in a negligible manner, as will be shown in the full version.) Let the 
elements of the field be {a1, ag,..., Gp}, and let the vertices V of the graph be 
labeled by the first n elements {a1, @2,...,@,}. Let S be any pre-specified sub- 
set of F with |S| = [a|F|/M]. To get a sample, generate w = (£0, £1,..., Zt—1) 
uniformly at random from F*, and define Y; to be 1 if ae rjal lies in S, and 
Y; = 0 otherwise. By construction, we have Pr |Y; = 1] ~ a/M; furthermore, it 
is well known that the Y; generated thus are all t-wise independent. Note that 
the above distribution is generating n coin tosses Y;, while we need only |D] of 
them; however, we can just ignore all the Y; for j ¢ D. 


Theorem 4. Assuming that the random marking is done using the t-wise in- 
dependent random variables explained above, or using independent random vari- 
ables, then the function € is cross-monotone, i.e., &(A,t) < &(D,1) for any 
ACD. 


Proof. The particular type of t-wise independent distribution that we use has 
the following crucial property: for any choice of w € Ft, if the set of marked 
demands in the run of SimpleCFL on A is A’(w) and the set of marked demands 
in the run on Dis D’(w), then A’(w) C D'(w). Define the joint probability 
p(F, E) to be the probability of selecting an element of the sample space w such 
that the marked demands in the run of SimpleCFL on A is F and on D it is E. 
Note that if p(F, E) > 0, then F C E. A joint probability with this property can 
also be defined for the case when the marking is fully independent. With this 
additional property, we can prove that the scheme € is cross-monotone. To see 
this, note that 


E(A,i) = 4 Dre P(F, E)[M éusr(F,%) + CG, F) 
> 3 VreP(L, F)[M Emsr(E,t) + £(é, E)] = €(D, 4), 


where the inequality uses the fact that the support of p(F, E} is only on pairs 
with F' C E, and the cross-monotonicity of €ysr. 
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4 A New Analysis of the Performance of SimpleCFL 


Suppose first that the marking is done fully independently in Step (S1); we 
now give a way of analyzing SimpleCFL that is different from the one of [4]. We 
will then use this proof in Section 5 to show that our estimate of 8 changes 
negligible when we conduct a t-wise independent marking fro a high enough 
value of t. Recall that a solution to the Connected Facility Location Problem 
with demands D is given by the facilities F and the Steiner tree on them, with 
the cost being ` ;ep 4(9,1(9)) + Me(T). Of the two terms in the sum, the former 
cost is referred to as the connection cost, and the latter is the Steiner cost. 
Let OPT be an optimal solution with facilities F* and tree T*, and let C* and 
S* = M c(T*) be the connection and Steiner costs in OPT; also let Z* = C*+S*. 

The bound for the Steiner cost C is the same as in [4]: we consider the optimal 
solution OPT, and compute the expected cost assuming that we construct the 
Steiner tree in Step (S2) using the paths of the optimal solution; the proof is 
omitted here. 


Lemma 3. The expected cost of Step (S2) of SimpleCFL is at most 
S = E[Me(T)} <2(S*+ ac"), (4.2) 


Massaging the Optimal Solution: Before we proceed to bound the connec- 
tion costs, let us modify OPT to get another solution OPT’, which costs more 
than OPT, but which has some more structure that allows us to complete the 
argument. The new solution has the following properties: 


P1. OPT’ opens facilities at F’ C F*, and these are connected by a cycle T" 
instead of a tree T*; each demand j is now connected to the facility i'(j), 
which is not necessarily its closest facility. 

P2. The number of demands assigned to a facility in F’, except perhaps the root 
r, is a multiple of M. (We will call this the mod-M constraint). 


Lemma 4. There is a solution OPT’ with the properties above, which has con- 
nection cost C” < C* + S* and Steiner cost S < 2S*. Hence, the total cost of 
OPT’ is Z! < C* +39*. 


Proof. (Sketch) Modifying the assignment of demands to nodes in F* to satisfy 
the mod-M constraint is fairly simple. Given the tree T*, we process this in a 
two-pass fashion. The first pass goes bottom-up, starting at the leaves which 
must all be in F*, and making sure that each node satisfies the mod-M property 
by sending some 6 < M demand from the node to its parent in the tree. OPT 
has nodes that are not in F*, and hence cannot be facilities in F”. We eliminate 
these nodes during a top-down pass, where we distribute their demand among 
their children. 


Back to Bounding the Connection Cost: From now on, we shall only con- 
sider the modified solution OPT’. For simplicity, by making copies of nodes ifnec- 
essary, we can assume that each node in F” has been assigned M demands. Recall 
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Fig. 1. The transformed instance OPT’ 


that T” is a cycle which contains r. (See Figure 1.) Starting from r, let us name 
the facilities on T” in (say) clockwise order r = fo, f1,..-, fe, fk+41 = r. (Hence 
referring to fı makes sense for all l, since we can just consider l mod (k + 1).) 

Let D; be the set of demands assigned to f;, and hence |D;| = M for l 4 0 by 

assumption (P2). Let us abuse notation and add r to Do. Let Pf be the portion 
of the cycle T’ joining fı and fi41, and let c(P/) denote the length of Pf, and 
hence S’ = M Da c(P;). Let Cy be the total connection cost of the demands 
in D; in the solution OPT’ (with C’ = -G C7). Our algorithm chooses, in 
Step (S3), the cheapest assignment of demands to nodes in F. Thus, to bound 
the expected connection cost, it suffices to bound the expected connection cost 
of an arbitrary experiment that assigns demands to nodes in F. We present one 
such experiment next and analyze it; an improved experiment will be presented 
in the final version of the paper. 
The Candidate Assignment: We now consider the following experiment: if Dı N 
F Æ Q, then we assign all the demands in D; to the element in Dı N F that is 
closest to fı. Else we have to assign these demands to some other open facility 
(i.e., “go outside Dr” ). In this case, consider the smallest t such that Djs,NF Æ 9; 
let s = l+ t. Note that r € Do, and so t < k. We now send all the demands in D; 
to the facility in D, closest to f,. If we assign demands in D; to a marked node in 
Dışı, then a path for each demand in D; goes through fi, Pj, Phro- o Praga 
and then from fii+ to the element in Dijiz N F closest to it. We bound the 
expected cost of these paths, which in turn bounds the expected connection cost 
of SimpleCFL. 

Let X; be the indicator variable that D; N F = @ in our algorithm. (Note 
that Xo = 0 with probability 1.) Let A; be the distance from f; and the closest 
element of D;N F. (If this is empty, A; can be set to 0.) By the above arguments, 
the assignment cost of the M demands in D; is at most 


C! +M (XX) e(P) + M (Xr Xin) —-Xi) A (4.3) 
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Indeed, the first term is the distance traveled by the demands in D; to reach 
fi, the second term expresses the fact that demands use FP; (and pay M c(P/)) 
if and only if Dı,..., D; all do not intersect F, and the third term implies that 
we incur a cost of M A; if we assign demands in D; to the closest member of 
D,OF. 

Note that X; and X; are seen for i # j, and that E[X;} = (1 - 
a/M)“ = q. To boina M - Ef (1 — X;)A;], let the distances from f; to the 
members of D; be ai < ag < ... < am with yay a; = C;. Now, 


E/(1-xX NA] = 5 ,a; x a/M x (1—a/M))- 


Note that the coefficients of a; decrease as j increases, and hence subject to the 
constraints above, the expectation is maximized when all the a;’s are equal to 
C!/M. This yields E[(1—X;)A;] < (C{/M) [1—-(1—a/M)™] = (C{/M) (1—q). 
Let Cı be the expected connection cost of the demands in Dı in our current 
experiment. Combining the above-seen facts with the inequality (4.3), we get 
thatfor l > 0, 


Cı <Ci +M Da e(P!) q7 Hi yy iC (Q-a qg. (4.4) 


Note that Co = Cġ; adding this to the sum of (4.4) over all l, the total expected 
connection cost is 


C < Ero 0 +M Din (Pl) Dimna H + Dia Ci- a) Dia (4.5) 
<O +M Dia (P) + Di Cl- a) py S20 H 7S’ (4.6) 


However, q < 1/e%,and hence the second term is at most 1/(e% — 1). Now 
using Lemma 4 to replace C” by C* + S* and S’ by 2S*, we see that the expected 
connection cost of our algorithm can be bounded by C < 2 C*+ oes oO one 
with (4.2), the total expected cost is at most C+ S < 2(1+a)C*+(2+£ za 2) -S™. 
Since the optimal cost is Z* = C* + S* we choose a % 1.35 to minimize the 
approximation ratio, and get the following result. 








Theorem 5. SimpleCFL is a B = 4.7-approximation algorithm for CFL. 


We will present an improved bound of a 4.6 approximation in the final version of 
the paper; this improved algorithm can also be derandomized in a fashion similar 
to that given in Section 5. If we were to use an 1.55-approximation for Steiner tree 
[10] to buy edges in Step (S2), we would be getting an improved approximation 
ratio of 4.2 for CFL, while the analysis of [4] gets a 3.55 approximation. However, 
we need to use the 2-approximation algorithm of the MST, as there is a cross- 
monotone cost-sharing function émsr for the MST problem. 


5 Analysis of the Limited-Independence Marking 


Let e be an arbitrary positive constant lying in (0,1). We now prove that if 
the demands are marked in t-wise independent fashion where t = alog(1/e) for 
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a suitably large constant a, then the expected approximation ratio is at most 
(1 +€) times what it is with independent marking. 

Recall that the total connection cost is a random variable that is the sum of 
three quantities: (i) the deterministic value $}, C], which represents the cost of 
all demands in each D; first getting routed to fy; (ii) the value that corresponds 
to unsuccessfully traveling through some D;, and (iii) the total cost paid in 
traveling from f; to the closest marked demand in D;, once D; is identified as 
the closest cluster. We will only show that the expected total cost of (111) gets 
multiplied by at most (1+ €) due to our t = alog(1/e€)-wise independent marking; 
the argument for the term (11) is analogous, and, in fact, simpler. Specifically, 
we will show the following. Let i be the index of some arbitrary but fixed D;. 
We show that the expected value of the random variable 


$ = Vy MXjXji Xi (1 - Xi Ai 


gets multiplied by at most (1 + €). (Since i is fixed, we have not subscripted ¢ 
as i; this remark also holds for many other definitions below.) We next present 
some notation. 


— For each group Dj, let 2;,1, Z;,2,..., Z;,m be the respective indicator random 
variables for the demands in D; getting marked by our algorithm, when 
these demands are considered in nondecreasing order of distance from fj. 
(This ordering will be important only in the case where j = i.) 

— Let A be any set of ordered pairs {(j,k)}. Then, M(A) is the indicator 
random variable for the event “for all (j,k) € A, Zik = 0”. Also, S(A) 
denotes 75 nye4 Zi,k- (“N” stands for none, and “S” for sum.) 


The following lemma distills some results known from [1,2,11], to give two 
types of upper bounds on E[N(A)]. 


Lemma 5. Lett; and t> be any even positive integers such that t1,tg < t(recall 
that the marking is done in a t-wise independent manner). The following hold 
for any set A of ordered pairs {(j, k)}. 


(i) Let IE(s, A) be the random variable denoting the inclusion-exclusion ex- 
pansion of N(A) truncated at the sth level; i.e., 


TE(s, A) = $ p=0(71)" Daca: jAl=r LI Gjeyear Zink: 
Then, the inequality N(A) < IE(t,, A) holds always; also, 


B(IE(t, A)] <(1—1/Myl4l + (14L) ” (5.7) 


t 
(That is, E[LE(t,, A)] is at most (3) ` more than what it would be if 


the marking was done completely independently.) 
(ii) Let NCM (te, A) denote the “normalized central moment” Sedla m z 
Then, the inequality N(A) < NCM (ta, A) holds always; also, 


if te < |A|/M, then E[NCM (tg, A)} < 8 - (2t2)®?/2 - (|A|/M)-*2/2. (5.8) 
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Proof. (Sketch) The upper bounds that are claimed to hold always on N(A) 
in (i) and (11), easily follow from the fact that tj and tg are even. Bound (5.7) 
follows from [2]; bound (5.8) follows from [1] (see also [11] for similar bounds). 


We now show how to use Lemma 5 to upper-bound E [¢]. Let a1, @2,...,av 
be the distances of the demands in D; from f;, written in nondecreasing order. 
Then, expanding the “(1 — X;)A;” part of ¢, we see that 


b= Det MX; Xizi Xian [he eZee (1 - Zae). 


Fix k arbitrarily, and let zk = Zin [Ipi (1 — Zao) Xia Xj Xj4a-+ + Xi-1. We 
aim to show that E [zx] is multiplied by at most (1+) in our t-wise independent 
marking, as compared to the fully-independent marking. For 7 = 0,1,...,2—1, 
define A; = {(r,s): (@-j) <r <(i-1), 1 <s <M}, and Bik = {(i,4): 1< 
£ < k — 1}. Thus we get 


zk = Zik © N(Bik)- a N (Aj). 


Now let a; be a sufficiently large constant. Define ag to be the smallest even 
integer that is at least 2elog(1/e), œ to be a, log(1/e), tı to be the smallest even 
integer that is at least 2ea, log(1/e), and tz to be the largest even integer that 
is at most a; log(1/e)/4. Rewrite zę as 


Lik : N (Bik) ' ze N(A;) + Zik . N (Bix) . Sirsa N(A;). 


Thus, by Lemma5, zę is always bounded by the sum of the following two random 
variables: 


Lik : IE (ao, Bik) > pee IE(t, A;), and 

Deh * IE(az, Bik) : ae NCM (ta, A;). 
If we expand the expectations of these two random variables using the linearity 
of expectation, we get terms each of which is a product of some of the random 
variables Z.,.; importantly, the number of factors in each such term is at most 
l +az2+tı and 1+a2+¢2 respectively. Thus, if we choose t = 1+ a2 + tı (recall 

that tı > t2), then the expectations of these two random variables become 

E [Zik]: E LE (ae, Bi x)| - D E[IE(ti, A;)|, and (5.9) 
EZ; x] -E(LE(a2, Bik) - Ge E(NC'M (te, A;)] (5.10) 
respectively. We next use (5.7) and (5.8) to bound these values; we will see that 


choosing the constant a, large enough results in E[z,] being at most (1 + €) 
times what it is with independent marking. The expression (5.9) is at most 


fe (Gd (S) ) Essa (0-a G) 
<h a-p (E) Dyce (Cl Mi + (ea), 


a2 
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Similarly, the expression (5.10) is at most $ - ((1 — 1/M)*~! + (e/ag)*?) - 
> a(S * (2t2/ j)*/2). Thus, if the demands are marked in t-wise independent 
fashion, then E [zx] is at most # - ((1 — 1/M)*~? + (e/a2)°?) times 


X o<j<a ((1 — 1/M)™i + (ea, log(1/e)/t1)") + 0,5,,(8 « (2te/7)*?/?). 


On the other hand, under fully-independent marking, E [ze] = 4 -(1-1/M)*7?- 
5 j>o(ł—1/M \Mi, Recalling the definitions of a1, a2, tı and tz, it is easy to verify 
that if a; is chosen as a sufficiently large constant, then the former value is at 
most (1 + €) times the latter. 


Further Extensions: We can improve on Theorem 5 using the following ex- 
periment: for a D; with no open facilities, instead of going clockwise around the 
cycle in OPT’, we find the smallest t such that Di44 or D;_; has an open facility, 
and assign all demands in D; to this facility. This gives us a 4.6-approximation. 
For the model of connected facility location with facility opening costs, we can 
also get constant-approximate cross-monotone cost shares by an analysis similar 
to that given here. (The proofs are deferred to the final version of the paper.) 
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Abstract. In this paper we study the approximability of the maximiza- 
tion version of constraint satisfaction problems. We provide two proba- 
bilistic approximation algorithms for MAx kKCONJSAT which is the prob- 
lem to satisfy as many conjunctions, each of size at most k, as possible. 
As observed by Trevisan, this leads to approximation algorithms with 
the same approximation ratio for the more general problem MAX kCSP, 
where instead of conjunctions arbitrary k-ary constraints are imposed. 
The first algorithm achieves an approximation ratio of 214°~*. The sec- 
ond algorithm achieves a slightly better approximation ratio of DSE 
but the ratio is shown using computational evidence. These ratios should 
be compared with the previous best algorithm, due to Trevisan, that 
achieves an approximation ratio of 2'~*. Both the new algorithms use 
a combination of random restrictions, a method which have been used 
in circuit complexity, and traditional semidefinite relaxation methods. 
A consequence of these algorithms is that some complexity classes de- 
scribed by probabilistical checkable proofs can be characterized as sub- 
sets of P. Our result in this paper implies that PCP.,;[log, k] C P for any 
c/s > 2771+49, and we have computational evidence that if c/s > 2*—*** 
this inclusion still holds. 


1 Introduction 


In this paper we study the approximability of maximum k-constraint satisfaction 
problem (MAX kCSP). An instance of MAX kCSP consists of a set of Boolean 
variables and a set of weighted constraints, each acting over a k-tuple of the 
variables. The value of an assignment is the sum of the weights of the satisfied 
constraints. This problem was defined by Khanna et al. [9] and is a natural 
generalization of many well-known optimization problems. In fact it can express 
any MAX SNP problem. 

An algorithm r-approximates a maximum optimization problem if the ratio 
of the value of the solution returned by the algorithm, and the optimal value 
of the problem, is at least r. Trevisan observed in [12] that it is sufficient to 
r-approximate a restricted version of MAX kCSP in order to r-approximate 
MAX kCSP. The restricted version is called MAX kKCONJSAT and allows only 
constraints that are conjunctions of literals. The, to date, best approximation 
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algorithm for MAX kCONJSAT, and thereby also for MAX kCSP, is based on 
a linear relaxation and is due to Trevisan [12]. It achieves an approximation 
ratio of 2!~*. Since the work was first published in 1996, it has remained the 
best algorithm known and one could start to wonder if it, in fact, is the best 
possible polynomial-time approximation algorithm for MAX kCSP. In this work 
we provide two algorithms, with approximation ratios of 2'4°-* and 2!-54-*,, that 
show that this is not the case. The ratio of 21:54-* is shown using computational 
evidence rather than a proof. However, for the approximation ratio of 2!:4°-* 
we have a complete proof. The new approximation ratios can be considered as 
a moderate improvement, but are structurally interesting. 

In probabilistic proof checking, the problem of maximizing the accepting 
probability of a verifier looking at k bits can be described as an instance of MAX 
kCONJSAT [12]. Thus, our result in this paper implies that PCP, [log, k} C P 
for any c/s > 2571-40, and we have computational evidence that if c/s > 2*~1!:54 
this inclusion still holds. 

We point out that there exist better algorithms for MAX kCSP if the in- 
stances are satisfiable. In that case, another algorithm by Trevisan [13] achieves 
a ratio of glos(k+1)—F 

In another research direction, PCP techniques have been used to show that 
approximating MAX kCSP within certain ratios, in polynomial-time, cannot 
be done unless P = NP. A long line of results led to the work of Samorod- 
nitsky and Trevisan [11], which show that there cannot exist a polynomial- 
time 2?Vk+1i-(k+1) approximation algorithm for MAX kCSP unless P = NP. 
More recently Engebretsen and Holmerin [2] achieved an even stronger result 
showing that, for k > 3, there cannot exist a polynomial-time 2V2*~2+1/2-*- 
approximation algorithm. 


Semidefinite Relaxation. Let us give a brief introduction into semidefinite re- 
laxations in connection with approximation algorithms. The work of Goemans 
and Williamson [5] laid ground for a large number of works providing better 
approximation ratios for various constraint satisfaction problems. The idea is to 
relax each clause into a set of semidefinite constraints. The resulting semidef- 
inite program can be solved to any desired accuracy in polynomial-time. The 
solution is then rounded into a solution of the original constraint satisfaction 
problem. In some cases it is possible to analyze the rounding technique used 
and thereby obtain an approximation ratio. For MAX 2CONJSAT Goemans and 
Williamson [5] obtained a 0.79607-approximation algorithm. Later, Karloff and 
Zwick [8] introduced a methodology to create semidefinite relaxations for any 
constraint function and tools to analyze the rounding scheme in some cases. Us- 
ing these methods Zwick [15] obtained a 0.5-approximation algorithm for MAX 
3CONJSAT and Guruswami et al. [6] obtained a 0.33-approximation algorithm 
for MAx 4CONJSAT. We note that these algorithms outperform the random 
assignment algorithm with more than a factor of 3, 4 and 5, respectively. In 
contrast, the general MAX KCONJSAT by Trevisan, which is based on a linear 
relaxation instead of a semidefinite relaxation, beats the random assignment al- 
gorithm with a factor of 2. Unfortunately, applying the semidefinite relaxation 
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method on MAX kCONJSAT for k > 4 makes the analysis of the approximation 
ratio problematic. An interesting question that remains open is if the algorithm 
achieves an approximation ratio of 2!°&*-*, as it does for k < 4, but we simply 
do not know how to show it. 


Techniques Used. The main idea of both our MAX kCONJSAT algorithms is 
to make use of the fact that semidefinite relaxation methods work well for con- 
junctions of small size. We shrink the size of the conjunctions by using random 
restrictions, an often used method in circuit complexity that was introduced by 
Furst et al. [4]. The random restriction is done by assigning a random value 
to each variable with high probability. Many of the conjunctions are unsatisfi- 
able after this random restriction and are thrown away, but the effective sizes of 
the remaining conjunctions have decreased so they are tractable for traditional 
semidefinite relaxation methods. For the 2!°4—-* algorithm we use the above 
mentioned methodology of Karloff and Zwick and analyze its effect on the re- 
stricted instance. For the 2!:4°~* algorithm we instead use so-called gadgets [14] 
to transform the restricted instance of conjunctions into a MAX 2SAT instance 
which is solved using the algorithm of Feige and Goemans [3]. 

When analyzing the approximation ratio of our 21°4—* algorithm we have 
to rely on the assumption that we have found global optima of two continuous 
optimization problems, one in ten variables and one in six variables. We note 
that this situation is quite common for many works that analyze the effective- 
ness of rounding procedures of semidefinite programs [3,6—8,15]'. In the case of 
our 2!-4°-* algorithm we rely on the Feige-Goemans algorithm which has been 
rigorously analyzed by Zwick [16]. 


Probabilistic vs. Deterministic Algorithms. We note that both our algorithms are 
probabilistic and thus the approximation ratio is calculated using the expected 
value of the produced solution. We do not see any problem derandomizing them 
with the method of Mahajan and Ramesh [10], but we have not studied it closely. 
Alternatively, multiple independent executions of the algorithm guarantees that 
except with exponentially small probability a solution with value arbitrarily close 
to the one stipulated is produced. For some applications only a lower bound of 
the optimum value is needed and we note that both algorithms can be used to 
get such a bound in a deterministic way (see Sect. 6). 


2 Preliminaries 


We say that an instance is p-satisfiable if the ratio of an optimal solution and 
the total weight of all clauses is at least p. An algorithm A a-approximates a 
maximization problem if for all instances x of the problem val(A, x)/opt(z) > 
a, where val(A,z) is the value of A(x) and opt(z) is the optimal value of zx. 


1 7 3 ; TE 
Some of these results have been proved in a rigorous manner since the original 
publication and many of the assumptions are seemingly not as strong as our is, 1.e. 
involve less variables. 
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Equivalently, A is said to have an approximation ratio of a. For probabilistic 
algorithms val(A, x) is allowed to be an expected value over the random choices 
done by A. 

An instance of the MAX kCSP problem consists of a set {Cj,...,Cm} 
of clauses (or constraints) with associated weights {w1,..., Wm} and a set of 
Boolean variables X = {x1,...,2%n}. Each clause C; consists of a Boolean func- 
tion fi of arity k and a size k tuple of Boolean variables (z;,,...,2;,) where 
zi; E X. A solution is an assignment to X and the value of the solution is the 
sum of the weights of the satisfied clauses. A clause C; = (fi, (@i,,---, Zip )) 18 
satisfied if fj(xi,,..., Zi, ) iS true. 

An instance of the MAX KCONJSAT problem is a special type of MAX kCSP 
problem where each clause is a conjunction of literals from X. A literal is either 
a variable or a negated variable. 


3 A 21-40-*_A pproximation Algorithm for MAX kCONJSAT 


In this section we describe how our first MAX KCONJSAT algorithm works. The 
algorithm runs both a random assignment algorithm and Algorithm Al, depicted 
in Fig. 1, and returns the best solution. 


| Input: A set of n Boolean variables {z1,...£n} and a set of m conjunctions 


{Ci,...Cm} with weights {wi,... wm}. 





l. (Initialization) Set the parameter s := 2.56. 
2. (Random restriction) Set J := @. For each i := 1,...n do: 
— With probability s/k: put iin J, I := I U {i}. 
— With remaining probability: let 2; be assigned according to an unbiased 
coin flip. 


3. (Gadget) Set Cosar := 0. For each j := 1,...m do: If C} still is satisfiable 
given the already set variables and include at most five unassigned variables, 
apply the gadget to 25AT as explained in Sect. 3.2. Put the 2SAT clauses in 
Csat, each with weight wy. 

. (Solve Max 25AT) Apply the Feige-Goemans algorithm [3] on Csat and let 
it assign the remaining variables {2; her. 


Fig. 1. Algorithm Al: a MAX kCONJSAT algorithm. 


The random assignment algorithm assigns a random Boolean value to each 
variable in the instance. The expected value of the solution is at least a 2~*- 
fraction of the total weight of all clauses. Thus, if the instance is not 271-40- 
satisfiable then the random assignment algorithm is a 2!40-*-approximation. 

If the instance instead is 2~1:*°-satisfiable then we can show that Algorithm 
Al produces a 2!:40-*-approximation. The algorithm starts by applying a ran- 
dom restriction over the instance, thereby reducing the size of the conjunctions. 
The small conjunctions are then transformed, using gadgets, into a MAX 2SAT 
instance which is solved using the algorithm of Feige and Goemans [3]. When 
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proving the approximation abilities of Algorithm A1, the main idea is to relate 
the optima of the MAX kCONJSAT instance and the produced MAX 2SAT in- 
stance. Theorem 1 is a consequence of the combination of Algorithm A1 and the 
random assignment algorithm. The corollary follows by using the observation 
made by Trevisan in [12]. 


Theorem 1. MAX kCONJSAT has a polynomial-time 2!-4°-*-approximation 
algorithm. 


Corollary 2. MAX kCSP has a polynomial-time 2!:4°-*-approrimation algo- 
rithm. 


Next we describe the two main techniques used in the algorithm, random re- 
strictions and gadgets. Then we analyze the approximation ratio of the algorithm 
and thereby prove Theorem 1. 


3.1 Random Restrictions 


In the introduction it was mentioned that semidefinite relaxation methods works 
well on small conjunctions. A MAX kCONJSAT instance consists of length-k 
conjunctions and in order to reduce the length of these conjunctions, and thereby 
making the instance tractable for semidefinite methods, we use a technique called 
random restrictions. For each variable, either assign it a random value with 
probability 1 — s/k, or let it be unassigned with probability s/k. Each clause is 
originally a conjunction of at most k variables. Restricting a clause to the values 
set by the random restriction either makes the clause true (if all literals in the 
clause were made true by the restriction), false (if any of the literals in the clause 
were made false by the restriction) or it is still a conjunction but with size z (Gf 
all but z literals were set and made true by the restriction). 


3.2 Gadgets Reducing CONJSAT Clauses to 2SAT Clauses 


A gadget is a way to transform a constraint of one type into a set of constraints 
of another type. In [14] Trevisan et al. exhibit a gadget reducing a conjunction 
of three literals, X; \X2AX3 into 2SAT clauses. A 2SAT clause is a disjunction 
of literals of size at most 2. The gadget is Y, (~Y V Xj), (Y V X2), (AY V X3), 
where Y is a so-called auxiliary variable that is added. Note that if X;A X2AX3 
is true, then all four of the gadget’s clauses can be made true by setting Y 
to true. However, if X1 A X2 A X3 is false then we can never satisfy all four 
clauses, but always three of them by setting Y to false. For those familiar with 
the jargon of [14], this is a strict 4-gadget. In [14] this gadget was used to create 
a 0.367-approximation algorithm for MAX 3CONJSAT. 

Generalizing this gadget construction we can produce a gadget reducing a 
conjunction of size 1, Xi A... A X; into i + 1 2SAT clauses: Y, (~Y V Xj), 
...(4Y V X;). (For i = 1 we do not have to use a gadget because X, is already 
a 2SAT clause.) In Algorithm Al, after the random restriction has reduced the 
size of the conjunctions, we use the above gadget to transform conjunctions into 
2SAT clauses. The gadget is more effective for small conjunctions, and that is 
why the gadget is used only on conjunctions of size five or less. 
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3.3 Analysis of the Approximation Ratio 


If an instance is not 2—!-4°-satisfiable, then a random assignment will be a 


21.40-k_approximation. Thus, left to be shown is that Algorithm Al produce 
21.40—-k_approximations on instances that are 2~1:4°-satisfiable. 

The first phase of Algorithm A1 is to apply a random restriction, with s = 
2.56 (see Sect. 3.1). This yields an instance of fairly small conjunctions. On 
each such iCONJSAT clause containing at most five literals we apply the gadget 
mentioned above, which reduces the clause to 7 length-2 and one length-1 2SAT 
clauses. Thus we get a MAX 2SAT instance that can be solved using the MAX 
2SAT algorithm of Feige and Goemans [3] thereby deciding the variables that 
remained unassigned after the random restriction. 

The probability that an arbitrary conjunction of length k remains satisfiable 
after the random restriction and contains exactly z unassigned variables is 


n- (JO OY re, 


Let Wrot be the sum of all weights in the MAX kKCONJSAT instance. The expected 
sum of weights from length-1 clauses that an instance induce is Œœ1Wtot and the 
corresponding value for length-2 clauses 1s Œ&2Wtot, Where ay = 5 p; and 
Q2 = ye ip; are the expected number of 2SAT clauses, length-1 respectively 
length-2, induced from a single conjunction of length k. 

Let the value of an optimal assignment be wopt = pwtot, Where p > 27 
Such an assignment, restricted to the unassigned variables, satisfy all of the 
length-2 clauses and an expected p-fraction of the weights of the length-1 clauses 
in the MAX 2SAT instance. Thus, the expected weight of satisfied length-2 
clauses iS &2Wtot and the expected weight of satisfied length-1 clauses is pay wor. 

Using the MAX 2SAT algorithm of Feige and Goemans [3] we are guaranteed 
to obtain a solution of expected weight at least 21 pay Wrot + 62&2Wtrot, where 61 = 
0.976 and fy = 0.931. The use of gadgets ensures that if we let the solution decide 
the value of the unassigned variables in the original MAX kKCONJSAT instance, 
the weight of that assignment will have expected value at least 6; pa wot + 
BoQ2Wtot — H2Wtot + PoWtot. We can now express an approximation ratio: 


1.40 


By poy Wror + Bea2Wtot — A2Wtot + PoWrot spä 
ne = e — 
PWrot 


p *(1~— b2)az +p tpo . 


We see that p; — (2s)*(i!)~te-*2-*, when k — oo. Using s = 2.56 we get 
a, = 7.618-2-*, ag = 27.401 -27* and po = 0.0773. 2-* thus yielding an 
approximation ratio greater than 2!4°-* for values of k large enough. To show 
that the ratio is valid for all values of k > 5, we have numerically checked that 
the ratio is higher than 2!:4°~-* for all values of k from 5 up to 1000 and then 
obtained a lower bound of 2!-4°-* for all k > 1000 (details in the full paper). 
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4 A 2154-k. Approximation Algorithm for MAX kCONJSAT 


In this section we give another approximation algorithm for MAX kCONJSAT. 
This algorithm also runs two different algorithms: a linear relaxation algorithm, 
described in Sect. 4.1, and Algorithm A2, depicted in Fig. 2. The linear relax- 
ation algorithm performs well on instances that are (1 + 21!—*)/2-satisfiable and 
algorithm A2 achieves well on all other instances. 


Input: A set of n Boolean variables {t1,...%n} and a set of m conjunctions 
{C1,...Cm} with weights {w1,...wm}. 


. (Initialization) Set the parameters s := 2.8, P := 0.72. 

. (Semidefinite programming) Generate the semidefinite programming re- 
laxation of the input instance given in Fig. 3. Find an (almost) optimal solution 
Uo,...Un in polynomial-time using standard techniques. 

. (Random restriction) Set I := 0. For each i := 1,...n do: 

— With probability s/k: put iin J, J := I U {i}. 
— With remaining probability: let x; be assigned according to an unbiased 
coin flip. 

. (Hyperplane rounding / Random assignment) Choose a random hyper- 
plane normal ny € S”. With probability F do for each i € J: 

— If sgn(nx - vo) = sgn(mx# - v;) assign t; := 1. 

— If sgn(ny - vo) # sgn(ny - 4) assign x; := 0. 
With remaining probability let xv; be assigned according to an unbiased coin 
flip, for alli € T. 





Fig. 2. Algorithm A2: a MAX kCONJSAT algorithm. 


Algorithm A2 is similar in approach to Algorithm A1. The first step is to solve 
a semidefinite relaxation of the problem instance. As in the previous algorithm 
a random restriction is imposed on the variables (see Sect. 3.1), but instead of 
reducing the conjunctions of the unassigned variables into a MAX 2SAT instance 
we use the solution obtained from the semidefinite relaxation to assign values to 
the remaining variables by performing a random hyperplane rounding. 

As in many previous works dealing with approximation algorithms for MAX 
CSP problems using semidefinite programming we only present computational 
evidence for the approximation ratio. The result is valid if we have found the 
worst possible vector configuration in two different cases. More explicitly we rely 
on the following technical assumption. 


Assumption 3. Let P = 0.72, (vo,v1,...u3) be a vector configuration of four 
vectors in R* and z be the minimum of rela, where A C {1,2,3}, defined in Fig. 
3. Then 0.493 is a lower bound on 


(P. Ma [sgn(nh : vo) = sgn(np- v1) =... sgn(np-vs)]} + (1 — P): Sma 


for all possible vector configurations of (vo, v1, ... v3). 
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Furthermore, let (vo, v1, ...v4) be a vector configuration of five vectors in R°, 
and z be the minimum of rela, where A C {1,2,3,4}, defined in Fig. 3. Then 
0.337 is a lower bound on 


(P . wos [sgn(mp vo) = sgn (np v1) =... sgn(np_-v4)} + (1 — P)-27*)/z , 


for all possible vector configurations of (vo,U1,...V4). 


> Wis 
j=1 
z; Zrela, AC G;,|Al <4 


rely = 1 
ee 1 + vo- U 
—— 
L+vo-vitvo-vj +i +9; 
ee 
rele je} = min Lt Yeo + Yer + Yeo Ue, + Vor Voa 


(79,01,02,03)€ 4 
permute({0,1,5,k}) 


rely;y 


relij} = 


12 
4 


(90,91,09,03,074)€ 4 
permute({0,2,7,4,1}) 


Tel 5:5.k.1} mini 


) 


min 


S 0<i-< 2n 
=- litin 





Fig. 3. Semidefinite program relaxation for MAx kCONJSAT. 


We have computational evidence that vector configurations of four vectors 
where all pairs of vectors have angle 1.438 are global minima and attain values 
close to the lower bound of 0.493. Vector configurations of five vectors are defined, 
up to rotation, by ten pair-wise angles. Let ĝ;; be the angle between v; and vj. 
If for some a, 64; = 1.855 for all i # a and ĝi; = 1.424 for all other pair-wise 
angles, then the vector configuration seems to be a global minimum and attains 
a value close to the lower bound of 0.337. 

Due to space considerations we omit the analysis of Algorithm A2, which 
primarily consists of semidefinite relaxation techniques due to Goemans and 
Williamson [5] and Karloff and Zwick [8]. The complete analysis and proof of 
the following theorem is given in the full version of this paper. The corollary 
follows in the same way as in the previous section. 
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Theorem 4. If Assumption 3 is true, then MAX kCONJSAT has a polynomial- 
time 21:54-*_ approximation algorithm. 


Corollary 5. If Assumption 3 is true, then MAX kCSP has a polynomial-time 
21.54-k _annroximation algorithm. 


¿From a practical standpoint Algorithm A2 seems a bit odd. First it solves 
a huge semidefinite program and after this only a small fraction of the variables 
are assigned values according to the solution. A more efficient order seems to be 
to do the random restriction first, as in Algorithm Al, and build a semidefinite 
program from the small fraction of conjunctions that still are satisfiable. This 
approach is in fact also realizable, but each weight in the semidefinite program 
has to be adjusted according to the length of its conjunction. Further details is 
given in the full version of this paper. 

Next, we give a description of the linear relaxation algorithm. After that we 
describe the semidefinite relaxation and the rounding method used in Algorithm 
A2. 


4.1 A Linear Relaxation Algorithm for MAX kCONJSAT 
with Threshold Rounding 


The linear program relaxation is identical to the one by Trevisan [12]. Each 
binary variable x; is relaxed to t; such that 0 < t; < 1. The clause C; has an 
associated variable z; in the relaxation. If z; occurs positively in C}, then z; < ti 
is a constraint in the relaxation, and if z; occurs negated then z; < 1 — t;. It is 
easy to see that an assignment to MAX kCONJSAT can be transformed into a 
valid assignment for the linear program with the same objective value. This is 
done by letting t; = x;. 

The difference of this algorithm compared with [12] is in how an assignment 
to the linear relaxed program is transformed into an assignment of the associated 
MAX kCONJSAT instance. In [12] the value of x; was decided by a flip of an 
unbiased coin with probability (k — 1)/k and only in the remaining case had 
the value of t; some impact on the value of x;. We instead make a threshold 
rounding by setting x; = 1 if t; > 1/2 and x; = 0 otherwise. 


Theorem 6. A (1+€)/2-satisfiable instance of MAX kCONJSAT, where e > 0, 
can be approximated in polynomial time within 2¢€/(1 + €) of the optimal value. 


The key observation for proving the above theorem is that if the relaxed 
value z; of a clause C} is strictly larger than one half, then C} will be satisfied 
by the threshold rounding. The proof is given in the full version of this paper. 


4.2 Semidefinite Relaxations 


Remember that a MAX CONJSAT instance consists of a set of conjunctions 
{C1,...,Cm} with weights {w1,..., Wm} and a set of variables {z1,..., £n}. In 
the semidefinite relaxation each variable x; corresponds to a vector v; on the unit 
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sphere S” in R”+t. A variable can appear negated in a conjunction and thus 
there is for each x; a vector vi4n = —v; which correspond to Z;. This is done 
only for notational convenience and we could replace vj47, by —v; everywhere. 
By somewhat abusing previous notation, a conjunction C; can be described as a 
subset of [2n] such that if x; is part of the conjunction then i € C} and if Zjis part 
of the conjunction then +n € C}. There is also a vector vo which represents true. 
Karloff and Zwick [8] introduced the so-called canonical semidefinite relaxation 
which specifies how to create the strongest possible relaxations from a specific 
constraint function. This construction could be applied on conjunctions of k 
variables, but we only use the canonical semidefinite relaxation of each subset 
of literals of size at most four. In Fig. 3 the complete semidefinite program that 
we use is specified. 

Hyperplane rounding is a popular method of obtaining a solution of the 
original problem from the solution of the semidefinite relaxation and is done as 
follows. Choose a random hyperplane in R™*?!, Set x; = 1 if v; lies on the same 
side of the hyperplane as vo does, and otherwise set x; = 0. We employ the 
hyperplane rounding as well, but first after making a random restriction. Thus, 
only the variables that remain unassigned after the random restriction get their 
value set by the hyperplane rounding method, all others are set by unbiased coin 
flips. 


5 Approximating MAX kCONJSAT for Small Values on k 


In both Algorithm Al and Algorithm A2 the random restriction parameter s 
was the same for all values of k. For specific values of k we can obtain better 
approximation ratios by using a larger value on s. This is the case for both 
algorithms. In Table | the obtained approximation ratios for MAX kCSP with 
the corresponding value on s are listed for 5 < k < 12. For the approximation 
ratios of Algorithm A2, the parameter P = 0.72 was used. We note that for k < 4 
there are known semidefinite relaxation algorithms achieving better ratios. 


6 Relations with Probabilistical Proof Checking 


We use the standard notation and definitions in connection with probabilistical 
checkable proofs, PCPs. For those unfamiliar with these, we refer to the work of 


Table 1. Approximation ratios a4; of Algorithm Al (combined with a random as- 
signment) and approximation ratios a2 of Algorithm A2 (combined with a linear 
relaxation with threshold rounding) on MAX kCSP for 5 < k < 12, with tailor-made 
values on s. 


Pr Ts To [io Tir [i 


FFgaa 42|1.4 
3.1 a6 a 28 28 2s 28 28) 





FF Tog anal 88I1.78 TAN: 65/1.6 
s  (34[|3.2|31 a4 30 30 3.0/3.0] 
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Bellare et al. [1] which also contains the history of PCPs. The complexity class 
PCP, (log, g] contains all languages that have a verifier with completeness e, 
soundness s, which uses only a logarithmic number of random bits and asks at 
most q (adaptive) questions. 

The following theorem due to Trevisan shows that an approximation algo- 
rithm for MAX kKCONJSAT implies that certain PCP classes are contained in P. 


Theorem 7 (Trevisan [12]). f MAX kCONJSAT is r-approximable for some 
r <1, then PCP. s[log, k} C P for any c/s > i/r. 


Both algorithms presented in this work are probabilistic, thus it is not imme- 
diate that the above theorem can be applied. By inspecting the proof of Theorem 
7, however, it is clear that only a lower bound of the value of an optimal solu- 
tion is needed. For the algorithm described in Sect. 4 such a lower bound can 
be obtained deterministically from the solution of the semidefinite program in 
Algorithm A2 and by performing the (deterministic) linear relaxation algorithm 
with threshold rounding. Thus, we get the following theorem. 


Theorem 8. /f Assumption 3 is true, then PCP,,,[log,k| C P for any efs > 
gk 1.54 


We now turn to the algorithm from Sect. 3.2 and primarily Algorithm A1. 
As with Algorithm A2 we do not need to perform the rounding step (in the 
Feige-Goeman algorithm), because solving the semidefinite program is enough 
to get a lower bound of the value of an optimal solution. But the random restric- 
tion that is applied before the semidefinite relaxation has to be derandomized. 
Therefore, we construct a polynomial size family of k-wise independent random 
restrictions. As each conjunction ofsize k is analyzed separately we get the same 
approximation ratio if the random restriction is chosen from a family of k-wise 
independent random restrictions. By applying Algorithm Al using each one of 
these random restrictions, it is guaranteed that at least one performs as well as 
the expected value over all different possible random restrictions. Thus, a lower 
bound on the value of the optimal solution, where the bound is at most a factor 
gk-1.40 smaller than the optimal solution value is achieved. 


Theorem 9. PCP,,,/log, k] C P forany c/s > 25714, 
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Abstract. We study the problem of minimizing the broadcast time for a set of 
processors in a cluster, where processor p; has transmission time €;, which is the 
time taken to send a message to any other processor in the cluster. Previously, 
it was shown that the Fastest Node First method (FNF) gives a 1.5 approximate 
solution. In this paper we show that there is a polynomial time approximation 
scheme for the problems of broadcasting and multicasting in such a heterogenous 
cluster. 


1 Introduction 


Networks of Workstations (NOWs) are an extremely popular alternative to massively 
parallel machines and are widely used (for example the Condor project at Wisconsin 
[17]) and the Berkeley NOW project [16]. By simply using off-the-shelf PC’s, a very 
powerful workstation cluster can be created, and this can provide a high amount of par- 
allelism at relatively low cost. Since NOWs are put together over time, the machines 
tend to have different capabilities and this leads to a heterogenous collection of ma- 
chines, rather than a homogenous collection, in which all the machines have identical 
capabilities. 

One fundamental operation that is used in such clusters, is that of broadcast (this is a 
primitive in many message passing systems such as MPI [1, 6, 8]). In addition it is used 
as a primitive in many parallel algorithms. The main objective of a broadcast operation 
is to quickly distribute the input data to the entire network for processing. Another sit- 
uation is when the system is performing a parallel search, then the successful processor 
needs to inform all other processors that the search has concluded successfully. Vari- 
ous models for heterogenous environments have been proposed in the literature. One 
general model is the one proposed by Bar-Noy et al [3] where the communication costs 
between links are not uniform. In addition, the sender may engage in another commu- 
nication before the current one is complete. An approximation factor with a guarantee 
of O(log k) is given for the operation of performing a multicast. Other popular models 
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in the theory literature generally assume an underlying communication graph, with the 
property that only adjacent nodes in this graph may communicate. 

Broadcasting efficiently is an essential operation and many works are devoted to 
this (see [18,9,10,4,5] and references therein). In addition, for emergency notification 
an understanding of how to perform broadcast quickly is essential. 

A simple model and algorithm was proposed by Banikazemi et al [2]. In this model, 
heterogeneity among processors is modeled by a non-uniform speed of the sending 
processor. A heterogenous cluster is defined as a collection of processors p1, po,...,Dn 
in which each processor is capable of communicating with any other processor. Each 
processor has a transmission time which is the time required to send a message to 
any other processor in the cluster. Thus the time required for the communication is 
a function of only the sender. Each processor may send messages to other processors in 
order, and each processor may be receiving only one message at a time. 

Thus a broadcast operation is implemented as a broadcast tree. Each node in the tree 
represents a processor of the cluster. The root of the tree is the source of the original 
message. The children of a node p; are the processors that receive the message from p;. 
The completion time of a node is the time at which it completes receiving the message 
from its parent. The completion time of the children of p; is c; + 7 - t;, where c; is the 
completion time of p;, t; is the transmission time of p; and 7 is the child number. In 
other words, the first child of p; has a completion time of c; + t; ( = 1), the second 
child has a completion time of c; + 2t; (7 = 2) etc. See Figure 1 for an example. 





(b) Optimal Solution 


Fig. 1. An example that FNF does not produce an optimal solution. Transmission times of pro- 
cessors are inside the circles. Times at which nodes receive a message are also shown. 


A commonly used method to find a broadcast tree is referred to as the “Fastest 
Node First” (FNF) technique [2]. This works as follows: In each iteration, the algo- 
rithm chooses a sender from the set of processors that have received the message (set 
S) and a receiver from the set of processors that have not yet received the message (set 
R). The algorithm then picks the sender from s € S so that s can finish the transmission 
as early as possible, and chooses the receiver r € R as the processor with the minimum 
transmission time in R. Then r is moved from R to S and the algorithm continues. The 
intuition is that sending the message to fast processors first is a more effective way to 
propagate the message quickly. This technique is very effective and easy to implement. 
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In practice it works extremely well (using simulations) and in fact frequently finds op- 
timal solutions as well [2]. However, there are situations when this method also fails to 
find an optimal solution. A simple example is shown in Figure 1. 

Despite several non-trivial advances in an understanding of the fastest node first 
method by Liu [13] (see also work by Liu and Sheng [15] in SPAA 2000) it was not 
well understood as to how this algorithm performs in the worst case. For example, can 
we show that in all instances the FNF heuristic will find solutions close to optimal? 

Liu [13] (see also [15]) shows that if there are only two classes of processors, then 
FNF produces an optimal solution. In addition, if the transmission time of every slower 
processor is a multiple of the transmission time of every faster processor, then again the 
FNF heuristic produces an optimal solution. So for example, if the transmission time of 
the fastest processor is 1 and the transmission time of all other processors are powers 
of 2, then the algorithm produces an optimal solution. It immediately follows that by 
rounding all transmission times to powers of 2 we can obtain a solution using FNF 
whose cost is at most twice the cost of an optimal solution’. However, this still did not 
explain the fact that this heuristic does much much better in practice. Recently, Khuller 
and Kim [12] showed that the FNF heuristic actually produces an optimal solution for 
the problem of minimizing the sum of completion times. This property is used to show 
that the FNF method has a performance ratio of at most 1.5 when compared to the 
optimal solution for minimizing broadcast time. In addition the performance ratio of 
FNF is at least 55, As a corollary of the above approximation result, it is shown that if 
the transmission times of the fastest $ processors are in the range [1... C] then FNF 
produces a solution with makespan at most Topr + C. Itis also shown that the problem 
is NP-hard, so unless P = NP there is no polynomial time algorithm for finding an 
optimal solution. 

It was conjectured in [12] that there is a polynomial time approximation scheme 
(PTAS) for this problem. In this paper we prove this conjecture. However, this algorithm 
is not practical due to its high running time, albeit polynomial. 


2 Problem Definition 


We are given a set of processors (p1, P2, . - -Pn ) and there is one message to be broad- 
cast to all the processors. Each processor p; can send a message to another processor 
with transmission time t; once it has received the message. Each processor can be either 
sending a message or receiving a message at any point of time. Without loss of gener- 
ality, we assume that tı < tg < ... < tn and tı = 1. Also we assume that pı has the 
message at time zero. 

We define the completion time of processor p; to be the time when p; has received 
the message. Our objective is to find a schedule that minimizes Cmax = max; c; where 
c; is the completion time of processor p;. In other words, we want to find a schedule 
that minimizes the time required to send the message to all the processors. 

We recall the following definition from [12]. 


' One approach to obtain a PTAS might be rounding to powers of (1+ €). However, this does not 
work since their proof works only when the completion times of all processors are integers. 
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Definition 1. We define the number of fractional blocks as the number of (fractional) 
messages a processor can send by the given time. In other words, given a time T, the 
number of fractional blocks of processor p; is (T — c;)/ti. 


Our proof makes use of the following results from [12]. 


Theorem 1. /12] The Fastest Node First algorithm maximizes the total number of frac- 
tional blocks for any value T. 


3 Approximation Scheme 


We now describe a polynomial time approximation scheme for the problem of perform- 
ing broadcast in the minimum possible time. Unfortunately, the algorithm has a very 
high running time when compared to the fastest node first heuristic. 

We will assume that we know the broadcast time T of the optimal solution. Since 
ti = 1, we know that the minimum broadcast time T is between 1 and n, and we can try 
all possible values of the form (1 + e)! for some fixed € > 0 andj = 1. nal eee | 
In this guessing process we lose a factor of (1 + €). 

Lete’ > 0 be a fixed constant. We define a set of fast processors F as all processors 
whose transmission time is at most e’T’. Formally, F = {p;|t; < €T}. Let S be the 
set of remaining (slow) processors. We partition S into collections of processors of 
similar transmissions speeds. For i = 1...k,define S; = {p,|e’T(1+’)*"' < t; < 
'T(1 + €')*} where kis [ERS]. 

We first send messages to processors in F using FNF. We prove that there is a 
schedule with broadcast time at most (1 + O(e€))T such that all processors in F receive 
the message first. We then find a schedule for slow processors based on a dynamic 
programming approach. 





Schedule for F: We use the FNF heuristic for the set F to generate a partial broad- 
cast schedule. Assume that the schedule for F has a broadcast time of Ten pr. In this 
schedule every processor p; € F becomes idle at some time between Try r — t; and 
TNF. 

We will prove that there is a schedule with broadcast time at most (1 + O(€))T 
such that all processors in F receive the message first, and then send it to the slow 
processors. The following lemma relates 7’py~ with the time taken by the optimal 
schedule to propagate the message to any |F| processors. 


Lemma 1. Jn any schedule, we need at least Tpnr — 2€'T time units to have |F| 
processors receive (any portion of) the message. 


Proof. We prove this by contradiction. In any schedule let C be the number of pro- 
cessors that have completely received the message by time ¢. In addition, let J; be the 
number of processors that have started receiving the message by time t. Suppose that at 
time T’ < Tryp — 2¢'T, wehave Cr: + Ir > |F|. First note that I, < Ci since each 
processor in J; is getting the message from exactly one (distinct) processor in C;. This 
means we should have that Cy, > | F|/2. 
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If a schedule is able to complete sending the message to at least |F|/2 processors 
by time T”, then the number of fractional blocks of this schedule is at least | F'|/2. Since 
FNF maximizes fractional blocks by Theorem 1, we claim that FNF also has at least 
|F|/2 fractional blocks by time T”. Let ty be the transmission of slowest processor in 
F. Notice that in additional time ts(< e¢’T) all processors that had started receiving 
the message must have finished receiving it. In additional time ts(< ¢’T’), FNF most 
certainly can double the number of processors that have received the message. Thus be- 
fore Try p, more than |F'| processors would have received the message; a contradiction 
since Tp yr is the earliest time at which the fastest |F| processors receive the message 
in FNF. 


Lemma 2. There is a schedule in which all processors in F receive the message no 
later than any processor in S and the makespan of the schedule is at most (1 + 3e')T. 


Proof. The main idea behind the proof is to show that an optimal schedule can be 
modified to have a certain form. Consider the set of processors of an optimal schedule 
that have received any portion of the message by time Tpy r — 2e€’7. This consists of 
some fast processors, F’ and some slow processors S’. Let F” = F \ F” and S” = 
S \ S’. Note that |F”| > |.S’| since |F’| + |.S’| < |F| (Lemma 1). In the FNF schedule, 
by time Tyr all processors in F = F’ U F” have the message. We can now have 
the processors in F” send messages to all processors in S$’. Since |F”| > |S’| each 
processor sends only one message and this will take additional time at most €T. By 
time Trpyr + €T, all processors in F’ U S’ certainly have the message in addition 
to the processors in F”. Notice that the optimal schedule now broadcasts the message 
to all remaining processors in additional time T — (Tpyr — 2e’T). Thus we can also 
finish the broadcasting in additional time T — (Try gr — 2€’T). The broadcast time of 
this schedule is at most Teyp + €T +T — (Tener — 2T) = (14+ 3e')T. 


Create AH Possible Trees of S: For the processors in S, we will produce a set S of 
labeled trees 7. A tree 7 is any possible tree with broadcast time at most T consisting 
of a subset of processors in S. Then we label a node in the tree as 2 if the corresponding 
processor belongs to S; (i = 1...k). We prove that the number of different trees is 
constant. 


Lemma 3. The size of S is constant for fixed € > 0. 


Proof. First consider the size of a tree T (that is, the number of processors in the tree). 
Let us denote it as |7 |. Since the transmission time of processors in S is greater than 
e’T, we need at least e'T time units to double the number of processors that received 
the message. It means that given a processor as a root of the tree, within time T we can 
have at most 21/€ processors receive the message. Therfore, T22" €. Now each 
node in the tree can have different label 1 = 1...k. To obtain an upperbound of the 
number of different trees, given a tree J we transform it to a complete binomial tree of 
size 21/€' by adding nodes labeled as 0. Then the number of different trees is at most 


(k dle 12" 


Attach 7 to F: Let the completion time of every processor p; € F be cj. Each pro- 
cessor p; in F sends a message to a processor in S every t; time unit. Therefore, a 
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fast processor p; can send messages to at most X; = 55] other processors. Let 
X = a ep Xj. Let us consider the time x; of each sending point in X. We sort those 
xi in nondecreasing order and attach a tree from S to each point (See Figure 2). Note 
that we can attach at most |X| trees of slow processors. Clearly |X| < n. 


We check if an attachment is feasible, using dynamic programming. Recall that we 


partition slow processors into a collection of processors $1, S2,...5%(k = Seui), 
Let s; denote the number of processors in set S;. We define a state s[j,n1,n2,... ng] 
(0 < j < |X], 0 < n; < 5;) to be true if there is a set of 7 trees in S that we can 
attach to first 7 sending points and the corresponding schedule satifies the following 
two conditions: 1) the schedule completes by time T and 11) exactly n; processors in 
S; appear in j trees in total. Our goal is to find out whether s[j, $1, $2, ... Sx] is true 
for some J, which means that there is a feasible schedule with makespan at most T. 
The number of states is at most O(n***) since we need at most n trees (|X| < n) and 
Si In. 

Now we prove that each state can be computed in constant time. Given s[j —1,...], 
wecompute s|j,71, n2, . ng} as follows. We try to attach all possible trees in S to z. 
Then s[j, 21, 2,... Ny] is true if there exists a tree 7’ such that the makespan of T” is 
at most T ~ z; and s[7 — 1,1 — m1, n2 — M1,- - -Nk — mg] is true where 7” has m; 
slow processors belonging to set S;. It can be checked in constant time since the size of 
S is constant (Lemma 3). 


Theorem 2. Given a value T, ifa broadcast tree with broadcast time T exists, then the 
above algorithm will find a broadcast tree with broadcast time at most (1+¢’)(1+3¢’)T. 





Fig. 2. Attach trees for slow processors to fast processors. 
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Proof. Consider the best schedule among all schedules in which processors in F receive 
the message first. By Lemma 2, the broadcast time of this schedule is at most (1+3e’)T. 
We round up the transmission time of p; in S; to e'T(1 + €’)* where i is the smallest 
integer such that t; < e'(1 + e’)'T. By this rounding, we increase the broadcast time 
by factor of at most 1 + e’. Therefore, the broadcast time of our schedule is at most 
(1 +e)(1 + 3e)T. 


Theorem 3. The algorithm takes as input the transmission times ofthe n processors, 
and constants €,¢' > 0. The algorithm finds a broadcast tree with broadcast time at 
most (1 + €)(1 + e’)(1 + 3€’)T in polynomial time. 


Proof. We try the above algorithm for all possible value of the form T = (1 + €} for 
7 Slee. al This will increase the broadcast time by factor of at most 1 + e. 
Therefore the broadcast time of our schedule is at most (1 + €)(1 + €’)(1 + 3e€’)T. 

For each given value (1 + €}, we find FNF schedule for processors in F (it takes at 
most O(n log nY) and attach trees of slow processors to processors in F, using dynamic 


programming. As we discussed earlier, the number of states is O(n**") and each state 
can be checked if it is feasible in O((k + 1)2?/") time, which is constant. Thus the 
running time of our algorithm is olll (nlogn + (k + 1)? 41 -n*+1) where 
k is [280/9]. 


log(1+e’) 


4 Multicast 


A multicast operation involves only a subset of processors. By utilizing fast processors 
which are not in the multicast group, we can reduce the multicasting time significantly. 
For example, suppose that we have m processors with transmission time tı and m more 
processors with transmission time ta where tı < t2. Let we want to multicast a message 
to all processors with transmission time tg. If we only use processors in the multicast 
group, it will take t2 -log m time. But if we utilize processors with transmission time t1, 
we can finish the multicast in tj - (log m + 1). Therefore, when tı < t2, the speed-up 
is significant. 


Theorem 4. We have a polynomial time approximation scheme for multicasting. 


Proof. Note that if an optimal solution utilizes k processors not in the multicast group, 
then those processors are the k fastest ones. Therefore, if we know how many processors 
participate in multicasting, we can use our PTAS for broadcasting. By trying all possible 
k and taking the best one, we have PTAS for multicasting. 
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Abstract. We consider centralized deterministic broadcasting in radio 
networks. The aim is to design a polynomial algorithm, which, given a 
graph G, produces a fast broadcasting scheme in the radio network rep- 
resented by G. The problem of finding an optimal broadcasting scheme 
for a given graph is NP-hard, hence we can only hope for a good approx- 
imation algorithm. We give a deterministic polynomial algorithm which 
produces a broadcasting scheme working in time O(D logn + log? n), 
for every n-node graph of diameter D. It has been proved recently [15, 
16] that a better order of magnitude of broadcasting time is impossible 
unless NP C BPTIM E(nf0°8!°8%)), In terms of approximation ratio, 
we have a O(log(n/D))-approximation algorithm for the radio broadcast 
problem, whenever D = NR(log n). 





1 Introduction 


A radio network is a collection of stations, equipped with capabilities of transmit- 
ting and receiving messages. Stations will be referred to as nodes of the network. 
The network is modeled as an undirected connected graph on the set of these 
nodes. An edge e between two nodes means that the transmitter of one end of e 
can reach the other end. Nodes send messages in synchronous steps (time slots). 
In every step every node acts either as a transmitter or as a receiver. A node 
acting as a transmitter sends a message which can potentially reach all of its 
neighbors. A node acting as a receiver in a given step gets a message, if and 
only if, exactly one of its neighbors transmits in this step. The message received 
in this case is the one that was transmitted by the unique neighbor. If at least 
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two neighbors v and v’ of u transmit simultaneously in a given step, none of 
the messages is received by u in this step. In this case we say that a collision 
occurred at u. It is assumed that the effect at node u of more than one of its 
neighbors transmitting is the same as that of no neighbor transmitting, 1.e., a 
node cannot distinguish a collision from silence. 

Broadcasting is one of the fundamental primitives in network communication. 
Its goal is to transmit a message from one node ofthe network, called the source, 
to all other nodes. Remote nodes get the source message via intermediate nodes, 
along paths in the network. A broadcasting scheme for a given network prescribes 
in which step which nodes transmit. In this paper we concentrate on one of the 
most important and widely studied performance parameters of a broadcasting 
scheme, which is its broadcasting time, 1.e., the number of steps it uses to inform 
all the nodes of the network. Broadcasting time is considered as a function of two 
parameters of the network: the number n of nodes, and the radius D, which is 
the largest distance from the source to any node of the network. (For undirected 
graphs, the diameter is of the order of the radius.) 

In this paper we study centralized broadcasting. All nodes of the network know 
its topology, i.e., they have a labeled copy of the underlying graph as input. 
Each node also knows its label. Labels of all nodes are distinct integers. For 
simplicity, we assume that these are consecutive natural numbers with source 0 
but this assumption is not essential. Thus, the scheme itself, although executed 
in a distributed way by nodes of the network, can be considered as designed 
centrally, using the same sequential algorithm at each node. The input of this 
algorithm is the graph representing the network, and the output is a sequence 
of sets of nodes which are to transmit in consecutive steps. It is important to 
distinguish between the running time of the algorithm producing a broadcasting 
scheme for each graph and the broadcasting time of the output scheme, for a 
given graph. 

Our goal is to design a deterministic algorithm, running in polynomial time, 
which produces a fast broadcasting scheme for any input graph. The problem 
of finding an optimal deterministic broadcasting scheme for any input graph 
(i.e. a scheme having the smallest possible broadcasting time for this graph) is 
NP-hard. Hence the best we can hope for is a good approximation algorithm. 


1.1 Related Work 


Centralized broadcasting in radio networks has been studied, e.g., in [6, 7, 18, 
15-17]. In [6] it was shown that the problem of finding an optimal deterministic 
broadcasting scheme for any input graph is NP-hard. In [7] the authors gave a de- 
terministic polynomial algorithm which produces a broadcasting scheme working 
in time O(D log*(n/D)), for any n-node graph of diameter D. Hence they got a 
O(log*(n/D))-approximation algorithm for the radio broadcast problem. In [18] 
the authors showed a method consisting in partitioning the underlying graph 
into clusters, which improves the time of broadcasting, since known broadcast- 
ing schemes can be applied in each cluster separately, and diameters of clusters 
are smaller than the diameter of the graph. Applied to the randomized scheme 
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from [3], working in expected time O(D logn + log? n), the method from [18] 
produces, for any n-node graph G of diameter D, a randomized scheme with ex- 
pected broadcasting time O(D-+log” n). Using the broadcasting scheme from [7], 
this method constructs (in polynomial time) a deterministic broadcasting scheme 
working in O(D + log® n) steps. 

Recently, the clustering method from [18] has been improved in [17]. Applied 
to the randomized scheme from [3], this improved method produces, for any n- 
node graph G of diameter D, a randomized scheme with expected broadcasting 
time O(D + log* n). On the other hand, using the broadcasting scheme from [7] 
this method constructs (in polynomial time) a deterministic broadcasting scheme 
working in O(D + log’ n) steps. On the negative side, it has been proved in 
[15] that o(log n)-approximation of the radio broadcast problem is impossible, 
unless NP C BPTIME(n©(8!0e"))_ (Here o(logn) is meant as a multiplicative 
factor.) Under the same assumption it was also proved in [16] that there exists a 
constant c such that there is no polynomial-time algorithm which produces, for 
every n-node graph G, a broadcasting scheme with broadcasting time at most 
opt(G') + clog? n, where opt(G) denotes optimal broadcasting time for G. 

There is also another approach to the problem of radio broadcasting, where 
the knowledge of the entire network topology is not assumed, and distributed 
broadcasting algorithms relying on limited information about the network are 
sought. It is not surprising that broadcasting under such scenarios is much slower 
than in the case of centralized broadcasting. Many authors [5,8—10,12,14] stud- 
ied deterministic distributed broadcasting in radio networks assuming that every 
node knows only its own label. In [8—10,14,20,13] the model of directed graphs 
was used. Increasingly faster broadcasting algorithms working on arbitrary n- 
node (directed) radio networks were constructed, the currently fastest being the 
O(n log? D)-time algorithm from [13]. (Here D is the radius of the network, i.e, 
the longest distance from the source to any other node). On the other hand, 
in [12] a lower bound {2(n log D) on broadcasting time was proved for directed 
n-node networks of radius D. 

Randomized broadcasting algorithms in radio networks were studied, e.g., 
in [3,13, 22]. For these algorithms, no topological knowledge of the network was 
assumed. In [3] the authors showed a randomized broadcasting algorithm running 
in expected time O(D logn + log? n). In [21] we improved this upper bound by 
presenting a broadcasting algorithm with expected time O(D log(n/D) +log? n). 
(Shortly later, a similar result was obtained independently in [13].) 


1.2 Our Results 


Our main result is a deterministic polynomial algorithm which produces a broad- 
casting scheme working in time O(D logn + log? n), for any n-node graph of di- 
ameter D. The method from [17], applied to our deterministic algorithm rather 
than to the randomized algorithm from [3], gives a deterministic polynomial al- 
gorithm producing a scheme with broadcasting time O(D +log* n). (Recall that 
the O(D + log* n)-time scheme, obtained in [17] by applying their method to 
the randomized algorithm from [3], was itself randomized.) The negative results 
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from [15,16] show that the order of magnitude of our broadcasting time cannot 
be improved, unless NP C BPTIM E(n0 “8 !o8”)), 

In terms of approximation ratios, we have a O(log(n/D))-approximation al- 
gorithm for the radio broadcast problem, whenever D = NR(log n). The best pre- 
vious approximation ratio for the radio broadcast problem was O(log*(n/D)), 
in [7]. It follows from the inapproximability result in [15] that we have the 
best possible order of approximation ratio, for D = R(logn), unless NP C 
BPTIM E(nO(slsn)), 


1.3 Terminology and Preliminaries 


The radio network is modeled as an undirected n-node graph G = (V,E). The 
source is denoted by 0. We denote by D the radius of the graph, 1.e., the largest 
distance from the source to any node of the network. Notice that, for undirected 
graphs, the diameter is of the order of the radius. Let £;, for 7 = 1,...,D, be 
the set of nodes of distance j from the source. L; is called the j-th layer. We 
denote by Ne(v) the set of neighbors of node v in graph G. For every node v 
in G, we define l, as the index of the layer containing node v (thus, we have 
v € Lı,). A simple path from node v € L; to node w € Ly; is called a straight 
path, if it intersects every layer in at most one node. A node w is called a straight 
predecessor of v in graph G if lẹ < l, and there is a straight path from w tov 
in graph G. Note that v is also its own straight predecessor. 
We will use the following lemma, proved in [7]. 


Lemma 1. Let H = (R, S, E) be an undirected bipartite graph. Suppose that all 
nodes in R have the source message, and no node in S has it. There exists a set 
Q C R such that if the set of nodes transmitting in a given step is Q then at 
least |S|/\In|S| nodes in S get the message after this step. Moreover, the set Q 
can be constructed in time polynomial in the size of H. 


In [7] the authors present an algorithm, polynomial in the size of H, to 
construct the set Q for a given undirected bipartite graph H. Iterating this algo- 
rithm 21n|S| times — after every step we remove all nodes in S which obtained 
the message — we get a scheme to inform at least |S|/2 nodes in S. Call this 
scheme CW. 

We will also use another scheme. Again consider an undirected bipartite 
graph H = (R, S, E), and suppose that all nodes in R have the source message, 
and no node in S$ has it. The simple round-robin scheme informs all nodes in S 
in time at most |S]. Call this scheme RR. 

We will use log instead of In in later definitions and calculations, where log 
denotes the binary logarithm. 


2 The Deterministic Algorithm FDB 


This section is devoted to the description of our deterministic algorithm FDB 
(Fast Deterministic Broadcasting) which produces a broadcasting scheme work- 
ing in time O(Dlog n+log? n), for any n-node graph of diameter D. Fix such an 
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input graph G. For simplicity of presentation we assume that n is a power of 2, in 
order to avoid rounding of logarithms. This assumption can be easily removed. 
The scheme is divided into phases, constructed recursively, each containing 3 
stages. 

In the first stage of phase O only the source transmits, and in remaining 
stages of phase O no action is performed. Hence after phase O all nodes in L, 
have the source message. For every phase k > 1 and layer number 7, wedefine 
sets Sk j, Rx; at the end of phase k — 1. For k = 1 we define (at the end of 
phase 0): 


ve S14 = Usi L; and Ria = Li; 
— Sı; = and R;,; = @ for layer number j > 1; 


Suppose that we have already constructed phases 2 = 0,...,k — 1, together 
with sets Si+1,;, Ri41,; for all layernumbers j. We show how to construct phase k 
with corresponding sets Sk+1,j; Rk+1,j- 

Let Hk, = (Rk, j, Sk j, Ek;) be the bipartite graph with sets of nodes Rx,;, 
Sk, j, where the set of edges E, ; is defined as follows: {v, w} € Ey; iff v € Rk j, 
w E Sk j and there is a straight path from w to v in graph G which is one of the 
shortest straight paths from w to any node in Rķ,;. Consider the execution of 
scheme CW on graph H;,;. Since Hy; need not be a subgraph of G, we call the 
execution of scheme CW on Hy; a virtual execution. During it a node v € Rx; 
virtually transmits a message (if the scheme CW schedules a transmission) and a 
node w € S;,; virtually receives a message from some node in Rx; (if it would 
receive it in scheme CW). We assume that each virtually transmitted message 
is the same for all nodes in Rx,;, and that no node in Sx, has this message 
prior to the execution of the scheme. Actual transmitting and receiving in the 
underlying graph G (as opposed to their virtual counterparts) is referred to as 
real transmitting and receiving, or simply transmitting and receiving. 

Phase k consists of three stages. First we present the generic scheduling 
scheme (GSS(k, 7)) forphase kand layer number j, which will be used in stage 1. 


Regular Substage (Lasts 2logn Steps): 
Run scheme CW on graph H;,; to get the set S C Sx; of nodes that virtually 
receive a message in this graph. Denote by Sj, ; the set Sx, \ S. 


Closing Substage (Lasts 4logn Steps): 

If j = (k + 1) —logn and (Skl < 4logn then 

apply scheme RR to the subgraph of Hpx,; 
induced by nodes Ry; and S% ; N D341. 

Sk+1,j = 9; 
Sk+1,j+1 = Se41,541 U Sk, 53 

else 
produce the sequence of empty sets of transmitters; 
Se4ig = Sk4oig U Sh 55 
Skyr jtt = Skt jti U Skj \ Sh, 5- 


176 Dariusz R. Kowalski and Andrzej Pelc 


Output: 
List of O(log n) virtual transmitter sets: concatenation of the list from the 
regular substage (obtained by applying scheme CW , and of the list from the 
closing substage (obtained by applying scheme RR or adding the list of empty 
sets). If the execution of scheme CW is shorter than 2logn or the execution 
of scheme RR is shorter than 4logn, fill the remaining steps by empty sets 
of transmitters, to provide synchronous time in execution of phases. 


Let Vk denote the set of nodes that have the source message at the end of phase 
k —1. Note that Vı = Ng(0) = Lı. Now we define phase k of the scheme. 


Stage 0: 
For every layer number j, initialize Sk+1,; =@, Retij = 0 
Stage 1: 


For r :=1to 3 do 
For all j =r mod 3 in parallel do 
run GSS(k, 7) (sets S,41,; were defined in this procedure) 
For every step of broadcasting, consider the union of virtual transmitter sets 
obtained for this step, over all layers 7. This is the set of (real) transmitters 
to be used by the scheme in this step. 


Stage 2: 


Given the set Vg and the list of transmitter sets from stage 1, let Vk+1ı be the 
set of nodes that have the source message at the end of this stage. Construct 
sets Ry41,; as follows: for every node w € Sx41,;, find an arbitrary node 
v € Lj such that there is a straight path from w to v, and add node v to 
Re41,;- 


Figure 1 shows phases O and 1 of the algorithm. Part (a) depicts phase O, 
part (b) shows virtual transmissions during phase 1, and part (c) shows new sets 
S2,1, 52,2 at the end of phase 1. 


Intuition. The reason of dividing Stage 1 into three phases is to avoid interfer- 
ence between transmitters from close layers. The intuitive meaning of sets Sx; 
is the following: every node of S;,; has the property that all its straight pre- 
decessors in layer L; already have the source message after phase k. Sets Rz,j, 
included in layer Lj, are intended to transmit the message to sets Sk, j. As we 
will show in Lemma 3, messages from nodes in Rx; will be received by nodes in 
Skj 0541 not only virtually but also in the real execution in graph G. 

The idea of procedure GSS is to push at least half of the nodes from Sk,; to 
Sk+1,j+1 In the regular stage, using protocol Cw, and to push all these nodes 
in the closing stage, using protocol RR, if only fewer than 4logn nodes are 
remaining. Nodes pushed out to the set $41,541, associated with the next layer 
j + 1, will be taken care of in phase k + 1. We will prove later that if a node v 
virtually receives the message then one of its straight predecessors really receives 
the message in graph G, hence a really informed predecessor of node v is now 
in layer 7 + 1, so we put node v in the set Sk+1,;+1. We will also prove that 
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Fig. 1. Phases O and 1 of algorithm FDB: transmissions, virtual receiving and corre- 
sponding sets $1,1, 82,1, 52,2. 


the set Sk,k+1—logn 1S small, so we can, using additional sets of transmitters in 
the closing substage, push all the nodes from this set to the set Sk+1,k+2—log n 
(not only half of them as in the regular substage). This is because, in the closing 
substage, we use protocol RR which guarantees that all nodes in set Sx,441-1ogn 
virtually receive a message. Since their informed predecessors are in the next 
layer now, they may be considered in the set corresponding to the next layer. 


Remark. The set Vii; is obtained by adding to the set Vi new nodes that receive 
the source message in graph G for the first time during phase k. Although sets 
of real transmitters used in the scheme are equal to sets of virtual transmitters 
output by procedure GSS, the fact of virtual receiving of a message by node 
w € S;, when scheme CW is executed on graph Hz,;, does not necessarily mean 
that node w gets the source message in the real execution of our scheme in graph 
G in phase k. For example, if w is not a neighbor of any node in Rk,;j, it does 
not get the source message, and consequently it is not added to Vx44. 


3 Analysis of the Broadcasting Time 


In this section we show that the broadcasting scheme, produced by our algorithm 
FDB for ann-node graph G of diameter D, has broadcasting time O(D logn + 
log? n). 
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Assume that n > 8 - otherwise centralized broadcasting is trivially done in 
constant time. Notice that scheme RR, working in time at most |S|, completes 
broadcasting in the closing stage, if |S| < 4logn. We will use this fact in the 
proof of Lemma 4 

Suppose that our scheme completes broadcasting in phase t. Consider phase 
k<t. 


Lemma 2. Consider sets of transmitters corresponding to the same time step 
in Stage l of the scheme, for a fixed r = 1,2,3. These sets are disjoint and the 
sets of their neighbors in G are also disjoint. 


Proof. This follows from the fact that, for a given r, such sets are subsets of 
layers at distance at least 3. 


Lemma 3. Ifnode v € Sk j, for j < ly, virtually receives a message during the 
regular substage of phase k then there is a straight predecessor of vin layer j +1 
which receives the source message in phase k. 


Proof. The proof of the lemma follows from the following invariant: 

(a) ifv € Skj, for 7 < ly, then a straight predecessor of v in layer j has the 
source message at the end of phase k — 1; 

(b) if v € Skj, for j < ly, virtually receives a message during the regular 
substage of phase k then there is a straight predecessor of v in layer j + 1 which 
receives the source message in phase k. 

The proof is by induction on phase k. For k = 1, only set S1, is nonempty 
among sets S;,;. Consider node v € $;,;. Let w E€ Lı be the straight predecessor 
of v. Node w has the source message at the end of phase 0 = k — 1 by definition 
of phase O (in phase O only the source transmits and every node in Lı gets 
the source message). This proves invariant (a). The arguments for part (b) are 
similar as in the induction step proved below. 

Suppose that the invariant holds for all positive integers k’ < k. We prove it 
for k. First we prove invariant (a) for k. We may assume that 7 > 1 (otherwise, 
invariant (a) follows from the fact that all nodes in Lı have the source message 
at the end of phase 0). Let w € L; be a straight predecessor of node v. Since 
v € Skj, for some phase k’ < k node v has been added to S;-41,; during phase k’. 
Let w’ denote a straight predecessor of v, w in layer L;_, - it exists since j— 1 < 
j = lw < ly and w is a straight predecessor of v. By the invariant (a) for k’ we 
have that w’ has the source message at the end of phase k’. By the invariant (b) 
for k’ we have that w gets the source message from w’ during phase k’, and 
consequently w has the source message at the end of phase k’ < k — 1. 

Now we prove invariant (b) for k. Consider sets of transmitters during the 
regular substages in phase k. (Recall that, to transmit in graph G, we use the 
same sets of transmitters as in virtual transmitting.) Suppose that node v € S, ; 
virtually receives a message from node w € Rx,; in step l of phase k. It follows 
that w € L; is a straight predecessor of v. Let z € Lj41 be the second node on 
the straight path from w to v. Hence node z really receives the source message in 
step l. Indeed, its informed neighbor w (the fact that w has the source message 
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at the end of phase k — 1 follows from invariant (a)) transmits in step l. No node 
from layers 7 + 1 and j + 2 is allowed to transmit in this step, in view of the 
division of Stage 1 into three time periods. Also, no other neighbor of z from 
Rx,; transmits, since all of them are also straight predecessors of v, and if one of 
them transmitted then v would not virtually receive the message in step ł. This 
completes the proof of the invariant and of the lemma. 


Lemma 4. For every k,j, such that j < (k+1)—logn < n, wehave |Sx41,;| = 0. 


Proof. The lemma follows from the following invariant, for every k,7, such that 
j<k+1<n+logn: 


INV(k, j) 


(a) Uic Sk+ril < n2 k. E ar for j > (k +1) — logn, 
(b) [Sk+1,5] = 0, for j < (k +1) -— logn. 


The proof of the invariant is by induction on k. It is omitted due to lack of space 
and will appear in the journal version. 


Lemma 5. If node v € L;, does not belong to any set Sk+1,j, Jor j = 1,..., 
jy — 1, at the end ofphase k, then v has got the source message by the end of 
phase k. 


Proof. The proof is by induction on phase k. For k = 0, the lemma follows from 
the definition of sets S;,;. Suppose that the lemma is true for phase k — 1 > 0. 
Consider phase k. 

If node v is not in any set S,;, for j = 1,...,3, — 1, then v received the 
source message by the end of phase (k — 1), in view of the inductive hypothesis. 
Suppose that v is in some set S%,;,,where 1 < jo < jy — 1, but not in any Sk+1,j, 
for j = 1,...,37, — 1. By the description of the scheme, v cannot belong to any 
Sk+1,j for 7 > jo + 1. It follows that v € Sk+1,jo+1 and jo +1 = jy. Two cases 
are possible: 


Case 1. jo 4 (k +1) — logn. 

Since v € Sx41,j941, it follows that v has virtually received the message during 
the regular substage of phase k. Consequently, using inequality jo < Jy, Lemma 3 
implies that the straight predecessor of v in layer Jọ + 1 = jy has got the source 
message by the end of phase k — 1. However, the only such node is v itself. This 
concludes the proof in Case 1. 

Case 2. jo = (k +1) — logn. 

In this case the closing substage is applied to nodes in jọ during phase k. Note 
that, by Lemma 4, we have |$,41,;,| = 0 and DA < 4logn. Hence the natural 
properties of round-robin scheme RR imply that node v € Sh, jo Pas got the source 
message by the end of phase k, since its predecessor does single transmission 
during scheme RR. 


Theorem 1. The broadcasting scheme, produced by the algorithm FDB for an 
n-node graph G of diameter D, has broadcasting time O(D logn + log? n). 
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Proof. Each phase of the broadcasting scheme lasts O(log n) steps. It remains to 
show that O(D+log n) phases are enough to complete broadcasting. By Lemma 4 
we get |Sk+1,;| = 0, for k = D~—1+logn and all 7 < D. By Lemma 5 this implies 
that all nodes get the source message by the end of phase D — 1 + log n, which 
completes the proof. 


Corollary 1. Algorithm FDB applied to the clustering method from [17] pro- 
duces a broadcasting scheme with broadcasting time O(D + log n), for every 
n-node graph G of diameter D. 


Corollary 2. Algorithm FDB combined with the clustering method from [17] 
yields a O(log(n/D))-approzimation deterministic algorithm for the radio broad- 
cast problem, whenever D = §2{logn). 


Proof. For D = O(log* n), we have O(log(n/D)) = O(log n), so our broadcasting 
time O(Dlogn + log*n), exceeding the lower bound D by a factor O(log n), 
is a O(log(n/D))-approximation in this range. For D = 2(log*n), we have 
O(D+log* n) = O(D), so we have a constant approximation ratio in this range.) 


Corollary 3. The broadcasting time of the scheme produced by the algorithm 
FDB has optimal order of magnitude, unless NP C BPTIME(n0(8!08”))_ 


Proof. Assume that NP C BPTIM E(n? log log w does not hold. If D = 
N(logn) then our broadcasting time is O(Dlogn), which cannot be improved 
because the result from [15] implies that o(logn)-approximation of the radio 
broadcast problem is impossible. If D = O(log n) then our broadcasting time is 
D = O(log? n). A better order of magnitude would give time opt(G) + o(log? n), 
precluded by the result from [16]. 


4 Complexity of the Algorithm FDB 


We showed that the algorithm FDB constructs a broadcasting scheme with 
broadcasting time O(D logn+log’ n), for every n-node graph of diameter D. In 
this section we estimate the running time of the algorithm itself. 

It follows from [7] that the construction of scheme CW on an m-node graph 
can be done in time O(m? log m). The construction of scheme RR can be done in 
time O(m). We now estimate the time to construct schemes used in one execution 
of procedure GSS. The regular and closing substages contain one execution of 
Cw for m = O(n), and one execution of RR for m = O(log n), respectively. Thus 
the time to construct these schemes is O(n? logn). The rest of the procedure 
takes time O(n), hence the total time of one execution of procedure GSS is 
O(n? logn). 

Hence we get the following estimate of the running time of algorithm FDB: 


Time of stage 0 in all D phases — O(D). 
Time of stage 1 in all D phases — O(D) executions of GSS and finding the union 
of O(D) disjoint sets of transmitting nodes — the total time O(Dn? log n). 
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Time of stage 2 in all D phases — For each node vin Sx41,;, we find the shortest 
straight path to layer L;. Either v € Sk ; and then this path is the same as 
it was, or v E€ Sk j;~-1 and the new straight path is the old one without one 
node in L;~1. The constructed sets Ry; are disjoint, hence the total time 
is O(n). 


Theorem 2. Algorithm FDB runs in time O(Dn? logn) for a give n-node graph 
G of diameter D. 


Remark. We stress again the distinction between the running time of algorithm 
FDB which constructs a broadcasting scheme for a given graph G, and the 
broadcasting time of the obtained broadcasting scheme for this graph. 


5 Conclusion 


We presented a deterministic polynomial algorithm which produces a broadcast- 
ing scheme working in time O(D logn + log? n), for any n-node graph of diam- 
eter D. Combined with the clustering method from [17], our algorithm yields a 
deterministic polynomial algorithm producing a scheme with broadcasting time 
O(D + logt n). It remains open if the additive term log*n can be improved. 
More precisely: what is the fastest broadcasting scheme that can be produced 
by a polynomial deterministic algorithm, given an n-node graph of diameter D? 
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Abstract. We study the speed of convergence to approximate solutions 
in iterative competitive games. We also investigate the value of Nash 
equilibria as a measure of the cost of the lack of coordination in such 
games. Our basic model uses the underlying best response graph induced 
by the selfish behavior of the players. In this model, we study the value 
of the social function after multiple rounds of best response behavior by 
the players. This work therefore deviates from other attempts to study 
the outcome of selfish behavior of players in non-cooperative games in 
that we dispense with the insistence upon only evaluating Nash equilib- 
ria. A detailed theoretical and practical justification for this approach is 
presented. We consider non-cooperative games with a submodular social 
utility function; in particular, we focus upon the class of valid-utility 
games introduced in [13]. Special cases include basic-utility games and 
market sharing games which we examine in depth. On the positive side 
we show that for basic-utility games we obtain extremely quick conver- 
gence. After just one round of iterative selfish behavior we are guaran- 
teed to obtain a solution with social value at least ł that of optimal. 
For n-player valid-utility games, in general, after one round we obtain a 
= -approximate solution. For market sharing games we prove that one 
round of selfish response behavior of players gives N(-+—)-approximate 
solutions and this bound is almost tight. On the negative side we present 
an example to show that even in games in which every Nash equilibrium 
has high social value (at least half of optimal), iterative selfish behavior 
may “converge” to a set of extremely poor solutions (each being at least 
a factor n from optimal). In such games Nash equilibria may severely 
underestimate the cost of the lack of coordination in a game, and we 
discuss the implications of this. 











1 Introduction 


Traditionally, research in operation research has focused upon finding a global 
optimum. Computer scientists have also long studied the effects of lack of dif- 
ferent resources, mainly the lack of computational resources, in optimization. 
Recently, the lack of coordination inherent in many problems has become an im- 
portant issue in computer science. A natural response to this has been to analyze 
Nash equilibria in these games. Of particular interest is the price of anarchy in 
a game [8]; this is the worst case ratio between an optimal social solution and 
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a Nash equilibrium. Clearly, a low price of anarchy may indicate that a system 
has no need for a single regulatory authority. Conversely, a high price of anarchy 
is indicative of a poorly functioning system in need of some regulation. 

In this paper we move away from the use of Nash equilibria as the solution 
concept in a game. There are several reasons for this. The first reason relates 
to use of non-randomized (pure) and randomized (mixed) strategies. Often pure 
Nash equilibria may not exist, yet in many games the use of a randomized 
(mixed) strategy is unrealistic. This necessitates the need for an alternative in 
evaluating such games. 

Secondly, Nash equilibria represent stable points in a system. Therefore (even 
if pure Nash equilibria exist), they are a more acceptable solution concept if it is 
likely that the system does converge to such stable points. In particular, the use 
of Nash equilibria seems more valid in games in which Nash equilibria arise when 
players iteratively engage in selfish behavior. The time it takes for convergence to 
Nash equilibria, however, may be extremely long. So, from a practical viewpoint, 
it is important to evaluate the speed or rate of convergence. Moreover, in many 
games it is not the case that repeated selfish behavior always leads to Nash 
equilibria. In these games, it seems that another measure of the cost of the lack 
of coordination would be useful. 

As is clear, these issues are particularly important in games in which the 
use of pure strategies and repeated moves are the norm, for example, auctions. 
We remark that for most practical games these properties are the rule rather 
than the exception (this observation motivates much of the work in this paper). 
For these games, then, it is not sufficient to just study the value of the social 
function at Nash equilibria. Instead, we must also investigate the speed of con- 
vergence (or non-convergence) to an equilibrium. Towards this goal, we will not 
restrict our attention to Nash equilibria but rather prove that after some number 
of improvements or best responses the value of the social function is within a 
factor of the optimal social value. We tackle this by modeling the behavior of 
players using the underlying best response graph on the set of strategy states. 
We consider (best response) paths in this graph and evaluate the social function 
at states along these paths. The rate of convergence to high quality solutions (or 
Nash equilibria) can then be measured by the length of the path. As mentioned, 
it may the case that there is no such convergence. In fact, in Section 4.2, it is 
shown that instead we have the possibility of “convergence” to non-Nash equi- 
libria with a bad social value. Clearly such a possibility has serious implications 
for the study of stable solutions in games. 

An overview of the paper is as follows. In section 2, we describe the problem 
formulations and model. In section 3, we discuss other work and their relation to 
this paper. In section 4, we give results for valid-utility and basic-utility games. 
We prove that in valid-utility games we obtain a s--approximate solution if 
each player sequentially makes one best response move. For basic-utility games 
we obtain a z-approximate solution in general, and a s-approximate solution if 
each player initially used a null strategy. We then present a valid-utility game 
in which every Nash equilibria is at least half-optimal and, yet, iterative selfish 
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behavior may lead to only O(+)-approximate solutions. In section 5, we examine 
market sharing games and show that we obtain §2(;+ )-approximate solutions 
after one best response move each. Finally, in section 6, we discuss other classes 
of games and present some open questions. 


2 Preliminaries 


In this section, we define necessary game theoretic notations to formally describe 
the classes of games that we study in the next sections. The game is defined as 
the tuple (U, {S;}, {a;()}). Here U is the set of players or agents. Associated 
with each player j is a disjoint groundset V;, and S}; is a collection of subsets 
of V;. The elements in the a groundset correspond to acts a player may make, 
and hence the subsets correspond to strategies. We denote player j7’s strategy 
by s; € S}. Finally, a; : 11,5; — R is the private payoff or utility function for 
agent 7, given the set of actions of all the players. In a non-cooperative game, 
we assume that each selfish agent wishes to maximize its own payoff. 


Definition 1. A function f : 2” —> R is a set function on the groundset V. 
A set function f : 2” — R is submodular if for any two sets A,B C V, we 
have f(A) + f(B) > f(ANB)+ f(AU B). The function is non-decreasing if 
F(X) < F(Y) for any X CY CV. 


For each game we will have a social objective function y : H1;5; — R. (We 
remark that y can be viewed as a set function on the groundset UV;.) Our goal 
will be to analyze the social value of solutions produced the selfish behavior of 
the agents. Specifically, we will focus upon the class of games called valid-utility 
games. 


Definition 2. Let G(U, {S;},{a;}) be a non-cooperative game with social func- 
tion y. G is a valid-utility game if it satisfies the properties: 


— y is a submodular set function. 

— The payoff of a player is at least equal to the difference in the social function 
when the player participates versus when it does not. 

— The sum of the utility or payoff functions for any set of strategies should be 
less than or equal to the social function. 


This framework encompasses a wide range of games in facility location, traffic 
routing and auctions [13]. Here, as our main application, we consider the market 
sharing game which is a special case of valid-utility games (and also congestion 
games). We define this game formally in Section 5. 


2.1 Best Response Paths 


We model the selfish behavior of players using an underlying state graph. Each 
vertex in the graph represents a strategy state S = (s1,$2,...,$n). The arcs in 
the graph corresponds to best response moves by the players. Formally, we have 
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Definition 3. The state graph, D = (V,€), is a directed graph. Each vertex in 
Y corresponds to a strategy state. There is an arc from state S to state S’ with 
label j if the only difference between S and S' is only in the strategy of player j; 
and player j plays his best response in strategy state S to go to S. 


Observe that the state graph may contain loops. A best response path is 
a directed path in the state graph. We say that a player 7 plays in the best 
response path P, if at least one of the edges of P is labelled i. Assuming that 
players optimize their best response function sequentially (and not in parallel), 
we can evaluate the social value of states on a best response path in the state 
graph. In particular, given a best response path starting from an arbitrary state, 
we will be most interested in the social value of the the last state on the path. 
Notice that if we do not allow every player to make a best response on a path P 
then we may not be able to bound the social value of a state with respect to the 
optimal solution. This follows from the fact that the actions of a single player 
may be very important for producing solutions of high social value. Hence, we 
consider the following models: 


One-round path: Consider an arbitrary ordering of players 21,...,2,. Path P 
is a one-round path if it starts from an arbitrary state and edges of P are 
labelled 71, 72,...,%, in this order. 


Covering path: A best response path P is a covering path if each player plays 
at least once on the path. 

k-Covering path: A best response path P is a k-covering path if there are k 
covering paths P1, P2,..., Pp such that P = (P1, Pe,..., Px). 


Observe that a one-round path is a covering path. Note that in the one-round 
path we let each player play his best response exactly one time, but in a cov- 
ering path we let each player play at least one time. Both of these models have 
justifications in extensive games with complete information. In these games, the 
action of each player is observed by all the other players. As stated, for a non- 
cooperative game G with a socialfunction y, we are interested in the social value 
of states (especially the final state) along one-round, covering, and k-covering 
paths. 


A Simple Example. Here, we consider covering paths in a basic load balancing 
game; Even-Dar et al. [2] considered the speed of convergence to Nash equilibria 
in these games. There are n jobs that can be scheduled on m machines. It takes 
p; units of time for job 7 to run on any of the machines. The social objective 
function is the maximum makespan over all machines. The private payoff of 
a job, however, is the inverse of the makespan of the machine that the job is 
scheduled on. Thus each job wants to be scheduled on a machine with as small 
a makespan as possible. It 1s easy to verify that the price of anarchy in this 
game is at most 2. It is also known that this game has pure Nash equilibria 
and the length of any best-response path in this game is at most n? [1]. In 
addition, from any state there is a path of length at most n to some pure Nash 
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equilibrium [12]. It may, however, take much more than n steps to converge to a 
pure Nash equilibrium. Hence, our goal here is to show that the social value of 
any state at the end of a covering path is within a factor 2 of optimal. So take 
a covering path P = (S1, S2,..., Sk). Let i* be the machine with the largest 
makespan at state S; and let the load this machine be L*. Consider the last 
job 7* that was scheduled on machine 7, and let the schedule after scheduling 
7° be Sı. Ignoring job 7*, at time ¢ the makespan of all the machines is at 
least L* — pj+. If not, job j* would not have been scheduled at machine i*. 
Consequently, we have 2< Enaiz m(L* — p;+). Thus, if OPT is the value of 
the optimal schedule, then OPT > ))1<;<, pj/m = L* — pjs. Clearly OPT > pj» 
and so L* = L — pj» + pj» < 20PT. 


3 Related Work 


Here we give a brief overview of related work in this area. The consequences 
of selfish behavior and the question of efficient computation of Nash equilibria 
have recently drawn much attention in computer science [8,7]. Moreover, the use 
of the price of anarchy [8] as a measure of the cost of the lack of coordination 
in a game is now widespread, with a notable success in this realm being the 
selfish routing game [11]. Roughgarden and Tardos [10] also generalize their 
results on selfish routing games to non-atomic congestion games. A basic result 
of Rosenthal [9] defines congestion games for which pure strategy Nash equilibria 
exist. Congestion games belong to the class of potential games [6] for which any 
best-response path converges to a pure Nash equilibrium. Milchtaich [5] studied 
player-specific congestion games and the length of best-response paths in this set 
of games. Even-Dar et al. [2] considered the convergence time to Nash equilibria 
in variants of a load balancing game. They bound the number of required steps 
to reach a pure Nash equilibrium in these games. Recently, Fabrikant et al. [3] 
studied the complexity of finding a pure strategy Nash equilibrium in general 
congestion games. Their PLS-completeness results show that in some congestion 
games (including network congestion games) the length of a best-response path 
in the state graph to a pure Nash equilibrium might be exponential. Goemans 
et al. [4] considered market sharing games in modeling a decentralized content 
distribution policy in ad-hoc networks. They show that the market sharing game 
is a special case of valid-utility games and congestion games. In addition, they 
give improved bounds for the price of anarchy in some special cases, and present 
an algorithm to find the pure strategy Nash equilibrium in the uniform market 
sharing game. The results of Section 5 extend their results. 


4 Basic-Utility and Valid-Utility Games 


In this section we consider valid-utility games. First we present results concerning 
the quality of states at the end of one-round paths. Then we give negative results 
concerning the non-convergence of k-covering paths. 
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4.1 Convergence 


We use the notation from [13]. In particular, a strategy state is denoted by 
S = {s1,S2,...,5x} E€ S. Here s; is the strategy of player i, where s; C V; 
and V; is a groundset of elements (with each element corresponding to an ac- 
tion for player i); Ø; corresponds to a null strategy. We also let S@ si = 
{S1,..-, 8:1, S$, $i41,---, Sk}, ie. the strategy set obtained if agent 7 changes 
its strategy from s; to si. The social value of a state S is y(S), where y is a 
submodular function on the groundset U;V;. For simplicity, in this section we 
will assume that y is non-decreasing. Similar results, however, do hold in the 
general case. 

We also denote by a;(S) the private return to player i from the state S, and 
we let y4, (S) = y(S Us) —7(S). Thus, formally, the second and third conditions 
in definition 2 are a;(S) > y; (S @0;) and $7, ai(5) < y(S), respectively. Of 
particular interest is the subclass of valid-utility games where we always have 
ai(S) = y: (S © hi); these games are called basic-utility games (examples of 
which include competitive facility location games). 


Theorem 1. Jn basic-utility games, the social value of a state at the end of a 
one-round path is at least 3 of the optimal social value. 


Proof. Let R = {01,...,0n} denote the optimum state, and let T = {t1,... tn} 


and S = {si,...,8n} be the initial state and final states on the one-round path, 
respectively; we assume the agents play best response strategies in the order 
1,2,...,n.So T? = {81,...,3j-1,¢;,...,tn} is a state in our one-round path 


P = {T =T',T?,...,T"t! = S}. Thus, by basicness and the fact that the 
players use best response strategies, we have }), a;(T*t") = S0i_, %4, (T @0:) > 
>i Yo, (T? © 0). It follows by submodularity that }9,a,(T***) > 0, yh (S U 
T? @ 0) > WM- SUT) > N) — yS) — yT). Moreover, by basicness, 
Koan a a 4 a 9 Oly a = 
Y(T? © 0:) = P ya (T © 0i) — Pu w (T 89) = E alT H) — Eu w (T ® 
0i). Let T? = {0),...,0:-1,ti,...,tn}. Then, by submodularity, y(S) — y(T) > 
pF a,(T*T*) = D Yi (" p 0;) = 2 att) aes (T). Hence, y(S) syy 
y(2) — (S) — 27(T). Since 7(S) > y(T), it follows that 3y(S) > OPT. 0 


We suspect this result is not tight and that a factor 2 guarantee is possible. 
Observe, though, that the above proof gives this guarantee for the special case 
in which the initial strategy state is T = 9. 


Theorem 2. In basic-utility games, the social value of a state at the end of a 
one-round path beginning at T = 0 is at least ł of the optimal social value and 
this bound is tight. O 


It is known that any Nash equilibria in any valid-utility game has value 
within a factor 2 of optimal. So here after just one round in a basic-utility game 
we obtain a solution which matches this guarantee. However for non-basic-utility 
games, the situation can be different. We can only obtain the following guarantee, 
which is tight to within a constant factor. 
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Theorem 3. In general valid- eee games, the social value of some state on 


any one-round path is at least = + of the optimal social value. 


Proof. Let y(2)=oPT and assume that y(t1,t2,...,tn) < ŻOPT. Again, agent 
t changes its strategy from t; to s; given the collection of ae ‘a 
ECEE E A E ban if at any state in the path P={T=T!,T?,...,7"t! = 
S} we have o,(Tit) > > =-oPT then we are done. To see this note that a; (Ti) > 
y(T*+!)—-q(T*90;) > 0, ‘since y is non-decreasing. Thusy(T*+!) > rhe j(T**1) 
> ane) > + OPT. Hence we have, y(¢1Us1,...,t;U8;, tiga,---, ta). (T) = 
2a y(siUti,... stjUSj tjt. tn) — (81 Uhr, z -s 8j-1Ubj_1, 05, t541)--- stn) 
$ Eja NT Usi) AP) < Ej NTD -ATH OB) < Lge oC) 
< 5- OPT. Consequently y(01 U t1 U s1,...,0; U ti U Si, Cipi U ti+1, -30n U tn) 
— y(t) U $1,.. ‘Ge D t a) > OPT — y(t U s1,.. th USt itl». se An) 
> OPT- y(S)—-Ł0PT > > =! opr. Thus, there isa j > i aa that yz, (TH) > 
I EU Sierat aa wight) Tats J OPT > 5~ OPT. Thero wenud 
obtain a; (TI+1) > s-OPT for some j > i. g 





4.2 Cyclic Equilibria 


Here we show that Theorem 3 is essentially tight, and discuss the consequences 
of this. Specifically, there is the possibility of convergence to low quality states 
in games in which every Nash equilibria is of high quality. 


Theorem 4. There are valid-utility games in which every solution on a k- 
covering path has social value at most 1 of the optimal solution. 


Proof. We consider the following n-player game. The groundset of player ¿ con- 
sists of three elements x;, x; and y;. Let X = U;x; and X’ = Ux}. We construct 
a non-decreasing, submodular social utility function in the following manner. 
For each agent 1 < į < n, we have 


o; 


We define y}, (5) in an identical manner. Finally, for each agent 1 <2 < n, we 


if SN(XuUxX’)=9 


otherwise 


let Yy: (5) = = 1, VS. Clearly, the social utility function y is non- decreasing. To see 
that it is submodilar. it suffices to consider any two sets A C B. If y,,(B) =1 
then y4, (A) = 1. This follows as BN(X UX’) = @ implies that AN (XUX = =f. 
Hence y}, (4) > Vz: (B), Vi,VA C B. Similarly y,,(A) 2 Yy (B), Vi, VA C B. 
Finally, y, (4) = y,(B) = 1, Vi,VA C B. It is well known that a function f is 
submodular if and only if A c B implies f;(A) > f;(B), Vj € V — B. Thus y is 
submodular. 

With this social utility function, we construct a valid utility system. To do 
this, we create private utility functions a; using the following rule (except for a 
few cases given below), where X; = SM (z; U 2). 

ous) = ft TER ian 
(XUX)NS] if yi É Si 
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In the following cases, however, we ignore the rule and use the private utilities 
given in the table. 


si]s2]3]-- ]sn—a]snJor(S)]e2(5)[aa(5)]---an—1(5)[on(5), 
£1\%2|/03|---|On—-1/On| O | 1 QO |---| 0 0 
+ 1On—1/On| li 





Observe that, by construction, $7; ai(S) = y($) for all S (including the excep- 
tions). It remains to show that the utility system is valid. It is easy to check 
that a;(S) > y(S) — y(S 6 0;) = ¥;(S ® @;) for the exceptions. So consider the 
“normal” S. If S; N (x; Uzi) = @, then a,(S) = lwhen y: E€ S; and a;(S) = 0 
otherwise. In both cases a;(S) = y/(S @ G;). If S; N (z; U x4) Æ 0 then 


2 if y; € S; and (S-S) N(XUX’) =ð 

KS @9;) = 1 if y; € S; and (S ~ S) N (XU X) £9 

gr ZAA if y; € S; and (S-S;)N(X UX) =9 

0 if y; É S; and (S-S) N(XUX)#0 

' i Xi 
Consider the first case. We have that aœ;(S) = 1 + TRU NAST = 


[Xi | = Xi — = : 
1+ [ROXNARIFICKOX ASS] = Lt rane = 2 = 7i(S @9;). It is easy to 


verify that in the other three cases we also have a;(S) > y;(S ® 0;). Thus 
our utility system is valid. It remains to choose which subsets of each players’ 
groundset will correspond to feasible strategies in our game. We simply allow 
only the singleton elements (and the emptyset) to be feasible strategies. That 
the set of possible actions for player i are A; = {0;,2;,2;, yi}. Now it is easy 
to see that an optimal social solution has value n. Any set of strategies of the 
form {y1, oe 09 Yi-1, Zis Yit+1,--- Yn}; where Zi © {x3, 0%, Yi}; l1<icn gives a 
social outcome of value n. However, consider the case of n = 3. From the start 
of the game, if the players behave greedily then we can obtain the sequence of 
strategies illustrated in Figure 1. The private payoffs given by these exceptional 
strategy sets mean that each arrow actually denotes a best response move by 
the labelled agent. However, all of the (non-trivial) strategy sets induce a social 
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Fig. 1. Bad Cycling. 


outcome of value 1, a factor 3 away from optimal. Clearly this problem general- 
izes to n agents. So we converge to a cycle of states all of whose outcomes are a 
factor n away from optimal. 0 


So our best response path may lead to a cycle on which every solution is 
extremely bad socially, despite the fact that every Nash equilibria is very good 
socially (within a factor two of optimal). We call such a cycle in the state graph 
a cyclic equilibria. The presence of low quality cyclic equilibria is therefore dis- 
turbing: even if the price of anarchy is low we may get stuck in states of very 
poor social quality! We remark that our example is unstable in the sense that 
we may leave the cyclic equilibria if we permute the order in which players make 
there moves. We will examine in more detail the question of the stability of cyclic 
equilibria in a follow-up paper. 


5 Market Sharing Games 


In this section we consider the market sharing game. We are given a set U of n 
agents and a set H of m markets. The game is modelled by a bipartite graph 
G = (H UU, E) where there is an edge between agent j and market i if market 
i is of interest to agent j (we write j is interested in market 7). The value of 
a market i € H is represented by its query rate q; (this is the rate at which 
market 2 is requested per unit time). The cost, to any agent, of servicing market 
t is Ci. In addition, agent 7 has a total budget B;. It follows that a set of 
markets s; C H can be serviced by player j if $ ies, Ci < Bj; in this case we 
say that s; represents a feasible action for player 7. The goal of each agent is to 
maximise its return from the markets it services. Any agent 7 receives a reward 
(return) A; for providing service to market i, and this reward is dependent upon 
the number of agents that service this market. More precisely, if the number of 
agents that serve market iis n; then the reward R; = ome Observe that the total 
reward received by all the players is equal to the total query rate of the markets 
being serviced (by at least one player). The resultant game is called the market 
sharing game. Observe that if a;(S) is the return to agent j from state S, then 
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the social value is y(5'}) = )/ ;ey a;(S). It is then easy to show that the market 
sharing game is a valid-utility game [4]. We remark that the subcase in which 
all markets have the same cost is called the uniform market sharing game; the 
subcase in which the bipartite graph G is complete is called the complete market 
sharing game. Note that in this game, the strategy of each player is to solve 
a knapsack problem. Therefore, in order to model computationally constrained 
agents, we may assume that the agents apply A-approximation algorithms to 
determine their best-response strategies. We then obtain the following theorems 
concerning the social value after one round of best responses moves. 


Theorem 5. In market sharing games, the social value of a state at the end 
of a one-round path is at least aH FT of the optimal social value (or at least 


EDL if the agents use A-approzimation algorithms). 


Proof. Let R = {01,..., 0n} denote an optimum state. Here oj © H is the 
set of markets that player j services in this optimum solution; we may also 
assume that each market is provided by at most one player. Let T = {t1,... tn} 
and S = {s,...,5n} be the initial state and final states on the one-round 
path, respectively. Again, we assume the agents play best response strategies 
in the order 1,2,...,n. So in step r, using a A-approximation algorithm, agent 
r changes its strategy from t, to sp; thus T” = {s1,...,8,,tp41,...,én} is an 
intermediate state in our one-round path P = {T = T°,T?,...,T” = S}. Let 
a;(S) be the return to agent j, then the social value of the state S = T” is 
WS) = Liye Q(T”) So we need to show that Vjey a3(T") > tropoa OPT- 
Towards this goal, we first show that 7(5S) = } jey aj(T”) 2 Wr jeu % (TÌ). 
We know that agent 7 does not changes its strategy from s, after step r. Therefore 
a market i has a nonzero contribution in y(S) if and only if market i has a nonzero 
contribution in the summation ` jeu Qj (Ti). For any market i, ifi appears in one 
of strategies in T” then the contribution of 2 to y(S) is qg;. On the other hand, at 
most n players use market t in their strategies. Consequently, the contribution of 
market 7 in the summation -jey a;(T?) is at most (1+5+$+. : +2)qi = Hagi 
It follows that -jey 0;(T") > #- jeu a3(T’), as required. We denote by T 
the summation jey 2% (TÍ). Next consider the optimal assignment 2, and let 
Y; be the set of markets that are in serviced by agent 7 in ø; but that are not 
serviced by any agent in T”, that is, Y; = 0; — Urey sr. Now y(S) is greater than 
the value of all the markets in Urey (6r — Yr) since these markets are a subset of 
markets serviced in T”. Hence, using the notation (Q) = J ico 4i to denote the 
sum of query rates of a subset Q of the markets, we have y(S) > $ cy q(or—Yr). 
Next we will prove that T > 1 Xorcy 4(Yr). Let y be the markets in Y; that are 
serviced in TŻ, that is, Y; = Y;—(s,U---Us,;Utj41U-+-Ut,). Then y is a feasible 
strategy for agent 7 at step 7, and thus, since player 7 uses a A-approximation 
algorithm, we have Aa;(T) > q(Yj). Therefore, AT > Vey a(Y7). 

Finally, we claim that 7 > -jey q(¥}’). To see this, consider a any market 
i € Y;' = Y; — Yj. Then market 7 is not in the strategy set of any agent in 
T”, but is in the strategy set of at least one player in TI. Therefore, somewhere 
on the path P after T? some player must change its strategy and discontinue 
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servicing market i. Let b; be time step such that T® is the first state amongst 
Ti+? ..., T” that does not service market i. Let M; = {i € H|b; = 7} be the set 
of markets for which b; = 7. It follows that Urey Y” (t) = Urey Mr. Notice that 
M, Ct, and no other agents service any market in M, at step r. It follows that 
a,(T?) > q(M,). Ta Seu = Pjev (Mj) < Dev 04(T3) =T. 
Hence we have, OPT = ) jey q(o;) < S, jeu 10; — Yi) + Vey NG) < WS) + 
AERC FATET < (1+ + 1)An)y(5). o 


Theorem 6. In market sharing games, the social value ofa state at the end of 
a one-round path may be as badas z of the optimal social value. In particular, 
this is true even for uniform market sharing games and complete market sharing 
games. 


Proof. Consider the following instance of a complete market sharing game. There 
are m = n markets, and the query rate of market zis qi = —eforalll<icn 
where is sufficiently small. The cost of market i is C; = 1+(n—i)efor2 <i<n 
and C4 = 1. There are n players and the budget of player j is equal to 1+(n—J)e. 
Consider the ordering 1,2,...,7 and the one-round path starting from empty 
set of strategies and letting each player play once in this order. The resulting 
assignment after this one-round path is that all players provide market number 
1 and the social value of this assignment is n —€. However, the optimum solution 
is for agent j to service market j giving an optimal social value of nHn — ne. 
Thus, the ratio between the optimum and the value of the resulting assignment 
is H, at the end of a one-round path. 

The bad instance for the uniform market sharing game is similar. There are 
n markets of cost 1 and query rates q; = | — € forall 1 < i < n where e€ is 
sufficiently small. There are n players each with budget 1. Player 7 is interested 


in markets 7,7 +1,...,n and in market 1. It follows that the social value of the 
assignment after one round is n — €. The optimal social value covers all markets 
and its value is nH,, — ne. Thus, the ratio is F after one round. o 


6 Conclusion and Open Problems 


In this paper, we presented a framework for studying speed of convergence to 
approximate solutions in competitive games. We proved bounds on the out- 
come of one round of best responses of players in terms of the social objective 
function. More generally, one may consider longer (but polynomial-sized) best- 
response paths, provided the problem of cycling can be dealt with. In acyclic state 
graphs, such as potential games (or congestion games), the PLS-completeness 
results of Fabrikant et al. [3] show that there are games for which the size of 
the shortest best-response path from some states to any pure Nash equilibrium 
is exponential. This implies that in some congestion games the social value of 
a state after exponentially many best responses might be far from the optimal 
social value. However, this does not preclude the possibility that good approxi- 
mate solutions are obtained when short k-covering paths are used. This provides 
additional motivation for the study of such paths. Here we may consider using 
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a local optimization algorithm and evaluating the output of this algorithm after 
a polynomial number of local improvements. 

The market sharing games are not yet well understood. In particular, it is 
not known whether exponentially long best-response paths may exist. Bounding 
the social value of a vertex at the end of a k-covering path is another open 
question. Goemans et al. [4] give a polynomial-time algorithm to find the pure 
Nash equilibrium in uniform market sharing games. Finding such an equilibrium 
is NP-complete for the general case, but the question of obtaining approximate 
Nash equilibria is open. 
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Abstract. The linear ordering problem is easy to state: Given a com- 
plete weighted directed graph, find an ordering of the vertices that maxi- 
mizes the weight of the forward edges. Although the problem is NP-hard, 
it is easy to estimate the optimum to within a factor of 1/2. It is not 
known whether the maximum can be estimated to a better factor us- 
ing a polynomial-time algorithm. Recently it was shown [NVO1] that 
widely-studied polyhedral relaxations for this problem cannot be used to 
approximate the problem to within a factor better than 1/2. This was 
shown by demonstrating that the integrality gap of these relaxations is 
2 on random graphs with uniform edge probability p = ovicen / n. In this 
paper, we present a new semidefinite programming relaxation for the lin- 
ear ordering problem. We then show that if we choose a random graph 
with uniform edge probability p = 4, where d = w(1), then with high 
probability the gap between our semidefinite relaxation and the integral 
optimal is at most 1.64. 


1 Introduction 


Vertex ordering problems comprise a fundamental class of combinatorial op- 
timization problems that, on the whole, is not well understood. For the past 
thirty years, combinatorial methods and linear programming techniques have 
failed to yield improved approximation guarantees for many well-studied ver- 
tex ordering problems such as the linear ordering problem and the traveling 
salesman problem. Semidefinite programming has proved to be a powerful tool 
for solving a variety of cut problems, as first exhibited for the maximum cut 
problem [GW95]. Since then, semidefinite programming has been successfully 
applied to many other problems that can be categorized as cut problems such 
as coloring k-colorable graphs [KMS98], maximum-3-cut [GW04], maximum k- 
cut [FJ97], maximum bisection and maximum uncut [HZ01], and correlation 
clustering [CGW03], to name a few. In contrast, there is no such comparably 
general approach for approximating vertex ordering problems. 

In this paper, we focus on a well-studied and notoriously difficult combi- 
natorial optimization problem known as the linear ordering problem. Given a 
complete weighted directed graph, the goal of the linear ordering problem is to 
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find an ordering of the vertices that maximizes the weight of the forward edges. 
Although the problem is NP-hard [Kar72], it is easy to estimate the optimum 
to within a factor of t: In any ordering of the vertices, either the set of forward 
edges or the set of backward edges accounts for at least half of the total edge 
weight. It is not known whether the maximum can be estimated to a better 
factor using a polynomial-time algorithm. Approximating the problem to within 
better than oe is NP-hard [NVO1]. 

The linear ordering problem is also known as the maximum acyclic subgraph 
problem. Given a weighted directed graph, the maximum acyclic subgraph prob- 
lem is that of finding the maximum weight subgraph that contains no cycles. 
The forward edges in any linear ordering comprise an acyclic subgraph and a 
topological sort of an acyclic subgraph yields a linear ordering of the vertices in 
which all edges in the acyclic subgraph are forward edges. 

Recently it was shown that several widely-studied polyhedral relaxations for 
the linear ordering problem each have an integrality gap of 2, showing that it is 
unlikely these relaxations can be used to approximate the problem to within a 
factor greater than l [NV01,New00].The graphs used to demonstrate these inte- 
grality gaps are random graphs with uniform edge probability of approximately 
gvlogn /n, where n is the number of vertices. For sufficiently large n, such a ran- 
dom graph has a maximum acyclic subgraph close to half the edges with high 
probability. However, each of the polyhedral relaxations studied provide an up- 
per bound for these graphs that is asymptotically close to all the edges, which 
is off from the optimal by a factor of 2. 

In this paper, we first present a new semidefinite programming relaxation 
for the linear ordering problem. A vertex ordering for a graph G = (V, E) with 
n vertices can be fully described by a series of n + 1 cuts. We use this simple 
observation to relate cuts and orderings. We derive a semidefinite program for 
the linear ordering problem that is related to the semidefinite program used 
in the Goemans-Williamson algorithm to approximate the maximum cut prob- 
lem [GW95]. We note that by using different objective functions, our semidefinite 
programming relaxation can be used to obtain semidefinite relaxations for many 
other vertex ordering problems. 

Second, we show that for sufficiently large n, if we choose a random directed 
graph on n vertices with uniform edge probability p = 4 (i.e. every edge in 
the complete directed graph on n vertices is chosen with probability p), where 
d = w(1), our semidefinite relaxation will have an integrality gap of no more 
than 1.64 with high probability. In particular, the graphs used in [NVOI] to 
demonstrate integrality gaps of 2 for the widely-studied polyhedral relaxations 
fall into this category of random graphs. The main idea is that our semidefinite 
relaxation provides a “good” bound on the value of an optimal linear ordering 
for a graph if it has no small roughly balanced bisection. With high probability, 
a random graph with uniform edge probability contains no such small balanced 
bisection. 
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2 Relating Cuts and Orderings 


Given an undirected weighted graph G = (V,E), the maximum cut (maxcut) 
problem is to find a bipartition of the vertices that maximizes the weight of the 
edges crossing the partition. In 1993, Goemans and Williamson used a semidefi- 
nite programming relaxation to obtain a .87856-approximation algorithm for this 
fundamental graph optimization problem [GW95]. The goal of the Goemans- 
Williamson algorithm for the maxcut problem is to assign each vertex i € V a 
vector v; E€ {1,—1} so as to maximize the weight of the edges (i, 7) such that 
Vi = Vj. 

A closely related graph optimization problem is the maximum directed cut 
(dicut) problem. Given a directed weighted graph G = (V, A), the dicut problem 
is to find a bipartition of the vertices — call these disjoint sets V; and V2 — that 
maximizes the weight of the directed edges (i,7) such that vertex 7 is in set Vj 
and vertex 7 is in set V2. Note that the edges in a directed cut form an acyclic 
subgraph. We can generalize the dicut problem to that of dividing the vertices 
into k labeled sets Vi, V2,..., Vk so as to maximize the weight of the edges (i, 7) 
such that vertex 2 is in set Vk and vertex j is in set Vp, and k < h. We call this 
the k-acyclic dicut problem. The linear ordering problem is equivalent to the 
n-acyclic dicut problem. 


2.1 A Relaxation for the Linear Ordering Problem 


We can generalize the semidefinite programming relaxation for the dicut prob- 
lem [FG95,GW95] to obtain a new semidefinite programming relaxation for the 
linear ordering problem. The basic idea behind this formulation is a particular 
description of a vertex ordering that uses n+1 unit vectors for each vertex. Each 
vertex i € V hasn+1 (n= |V|) associated unit vectors: v?, vi» v?,...u?. In an 
integral solution, we enforce that v? = —1, v? = 1 and that v? and pit differ 
for only one value of h, 0<h<n. Constraint (1) enforces that in an integral 
solution, v? and ae differ for only one such value of h. This position h denotes 
vertex 2’s position in the ordering. For example, suppose we have a graph G that 
has four vertices, arbitrarily labeled 1 through 4. Consider the vertex ordering 
in which vertex 2 is in position 7. An integral description of this vertex ordering 
IS: 


{uy vl, Ui; Us, wp = fal, I 1, 4, 3, 
{v9, vd, v2, va, ve} = {-1,-1, 1, 1, 1}, 
{uf vi, v2, v3, ve} = {-1,-1,-1, 1, 1}, 
{v?, vl, vz, vi, vt} = {-1,-1,-1,-1, 1}. 


Let G = (V, A) be a directed graph. The following is an integer quadratic 
program for the linear ordering problem. For the sake of convenience, we assume 
that n is odd since this simplifies constraint (2). By P(G), we denote the optimal 
value of the integer quadratic program P on the graph G. 
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(P) 


E Reh. E s — 
max ) ) wis (V; Ue aU, ee AE — v? t. uf) (1) 
iJjEA 1<h<l<n 


v? Us jue ae — u}. gS —y} Uy >0 Wig EV, hele fn] 
v.u =1 Vie V, hejn] 
v vwo =-l VieV 
Vv; vwo =l vie V 
ŞO uf v? =0 (2) 
i jEV 
elhel vi, h efn]. (3) 


We obtain a semidefinite programming relaxation for the linear ordering problem 
by relaxing constraint (3) to: vë € R”, Vi,h. We denote the optimal value of 
the relaxation of P on the graph G as Ppr(G). 


2.2 Cuts and Uncuts 


Suppose we have a directed graph G = (V, A) and we are given a set of unit 
vectors {u;} € R”, O < i < n. We will define the forward value of this set 
of vectors as the value obtained if we compute the value of the dicut semidefi- 
nite programming relaxation [GW95,FG95] using these vectors. Specifically, the 
forward value for this set of vectors is: 


1 
max ; ql — Yi: vj — Vo: vi + vo: vs). (4) 
ijEA 


In an integral solution for the dicut problem, there will be edges that cross the 
cut in the backward direction, 1.e. they are not included in the dicut. For a 
specified set of unit vectors, we can view the dicut semidefinite programming 
relaxation as having forward and backward value. We define the backward value 
of the set of vectors {v;} as: 


1 
max X, 7 (1— Vi: Vj — Yo Vj + vovi), (5) 
ijEA 


The difference between the forward and backward value of a set of vectors {v;} 
1S: 


Í 
` 5 (¥j vo — vi : vo). (6) 
ijEA 


Lemma 1. /f a directed graph G = (V, A) has a maximum acyclic subgraph of 
(4 + 6)|A| edges, then there is no set of vectors {vi} such that the difference 
between the forward and backward value of this set of vectors exceeds 2ô| A|. 
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Proof. We will show that given a vector solution {v;} to the semidefinite program 
in which the objective function is (6) and all the v; vectors are unit vectors, we 
can find an integral (i.e. an actual cut) solution in which the difference of the 
forward and backward edges crossing the cut is exactly equal to the objective 
value. If the difference of an actual cut exceeds 2ô| A|, e.g. suppose it is (26+€)|Al, 
then we can find an ordering with (5 + 6 + €/2)|A| forward edges, which is a 
contradiction. This ordering is found by taking the cut that yields (26 + e)|A| 
more forward than backward edges and ordering the vertices in each of the two 
sets greedily so as to obtain at least half of the remaining edges. 

Suppose we have a set of unit vectors {v;} such that the value of equation 
(6) is at least (26 + €)|A| = |A|. We will show that we can find an actual cut 
such that the difference between the forward and the backward edges is at least 
G\|A|. Note that vov; is a scalar quantity since vo is a unit vector that without 
loss of generality is (1, 0,0,...). Thus, our objective function can be written as 
HE A 5 (2; — z;)where 1 > z; > —1. We transform the z; variables into x; 
variables that range between O and 1 by letting z; = 22; — 1. Then we have that 
DIEA (zj — 2i) = Ž ijea (£j — Ti). This results in a linear program. We claim 
that an optimal solution to the following linear program is integral. 


> G20 (7) 
ijEA 
O<a,<1, We. 


To show this, consider rounding the variables by letting 7 be 1 with proba- 
bility x; and 0 otherwise. Then the expected value of the solution is exactly the 
objective value. However, note that the value of the solution cannot be less than 
the expected value, since then there must exist a solution with value greater 
than the expected value, which contradicts the optimality of the expected value. 
Thus, the integral solution obtained must have difference of forward and back- 
ward edges that is equal to the objective value (7). O 


We will also find a discussion of the following problem useful. Consider the 
problem of finding a balanced partition of the vertices of a given graph (i.e. each 
partition has size 5) that maximizes the weight of the edges that do not cross 
the cut. This problem is referred to as the max-5-uncut problem by Halperin 
and Zwick [HZO1]. Below is a integer quadratic program for the max-#-uncut 


problem. 


(T) 
Lope; 
wjEA 
y- Ui Uj = 0 
i, JEV 
U- Ue S Vie V 


v; € {1,-1} VieV. (8) 
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We obtain a semidefinite programming relaxation for the max-#-uncut problem 
by relaxing constraint (8) to: v; E R”, Vi. We denote the value of the relaxation 


of T on the graph G as TR(G). 

Lemma 2. Let G = (V, A) ande,6 be positive constants. Suppose the maximum 
acyclic subgraph of G is (§ + 5)|A|. If Pr(G) > (1 — Al, then Tr(G) > 
(1 — 2e — 26) Al. 


Proof. For each edge tj € A, we have: 


h £ h—1 £—1} h £—1 h—1 £ 
h<£ 
S (uP — uf?) - (uf — uf") < 
h<£ 
h h-1 f— 
5 (v — v; ) -o +) + (10) 
AS$L<B 
NSO (up — up") - (uf — uf") + (11) 
h>}e> $ 
h h—1 £ £—i 
Ci ae (v= Vj ) (12) 
h<3 >% 


For each edge, we refer to the quantity (9) as the forward value for that edge 
with respect to Pr(G). The same term summed instead over A > £ is referred to 
as the backward value of the edge with respect to Pr(G). We can simplify the 
terms above. Let v; = ve. 


= 1 
SO (oh — vb). (vf — vf!) = —(u; + vo) « (vj + vo), 


h<Be<2 A 
= Be 1 
(up — uf") - (vf — vs") = q(vo — vi)» (vo = v5), 
h>% l>¥% 
1 
5 Qh 5 (vi — o = zi + U9) - (vo — vj). 
h<Be>R 


Since Pr(G) > (1 — €)|A], we have: 


5 2 (v? — v? j (v5 — v 1) = > Elvo ~ v4) - (vo +04) < eļAl. 


WEA h> ess ijEA 


The above inequality says that the backward value of the vectors {v;} (i.e. quan- 
tity (5)) is at most the backward value of Pr(G). By Lemma 1, the difference 
of the edges crossing the cut in the forward direction and the edges crossing the 
cut in the backward direction is at most 26|A]. 
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> “(vi + wo) . (vo — vj) = `S E(w — vi) ; (vo + v;) = 


ijEA ijEA 


1 
y zi CUE vo) < 26|Al. 
ijEA 
This implies that the forward value cannot exceed the backward value by more 
than 26|A|. Thus, we can bound the forward value as follows: 


SO E oh oh) fof) = YF Flo- ws) (vot 04) <(€ + 26)/A], 


(jEAh< 2 l>} ijEA 


This implies that if we sum the quantities (10) and (11) over all edges in A, then 
the total value of this sum is at least (1 — 2e — 28)| A|. The sum of (10) and (11) 
taken over all the edges is: 


1+ uj, -0v; 
————_, 13 
ek (13) 


ijEA 
0 


3 Balanced Bisections of Random Graphs 


A bisection of a graph is a partition of the vertices into two equal (or with 
cardinality differing by one if n is odd) sets. We use a related definition in this 
section. 


Definition 1. A y-bisection ofa graph for y < ł is the set of edges that cross 
a cut in which each set of vertices has size at least yn. 


Suppose we choose an undirected random graph on n vertices in which every 
edge is present with probability p = 2d The expected degree of each vertex is 
2d and the expected number of edges is dn. We will call such a class of graphs 
Gp. 


Lemma 3. For any fixed positive constants €, y, ifwe choose agraph G € Gp on 
n vertices for a sufficientlylarge n with p = zd and d = w(1), then the minimum 
y-bisection contains at least (1 — €)y(1 — y)2nd edges with high probability. 


Proof. We will use the principle of deferred decisions. First, we will choose a 
yn, (1 — yìn partition of the vertices. Thus y(1 — y)n? edges from the complete 
graph on n vertices cross this cut. Then we can choose the random graph G by 
picking each edge with probability p = 2d The expected number of edges from 
G crossing the cut is p = (y(1 — y)n?) (24) = y(1 — y)2dn. For each edge in the 
complete graph that crosses the cut, we have the indicator random variable X; 
such that X; = 1 if the edge crosses the cut and X; = 0 if the edge does not 
cross the cut. Let X = Y Xj, i.e. X is the random variable for the number of 
edges that cross the cut. By Chernoff Bound, we have: 


e2 y —yj2dn 
Pr[X < (1 — e)y(1 —y)2dn)) < eo ITAS, 
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We can union bound over all the possible yy-bisections. There are less than 2” 
ways to divide the vertices so that at least yn are in each set. Thus, the prob- 
ability that the minimum 7-bisection of G is less than a (1 — €) fraction of its 
expectation is: 


n 


Pr[min y-bisection(G) < (1 — €)y(1 — y)2nd] < -aapa Eero 
e 3 


Let d = w(1). Then for any fixed positive constants ~y, €, this probability will be 
arbitrarily small for sufficiently large n. oO 


4 A Contradictory Cut 


In this section, we will prove our main theorem. Suppose we choose a directed 
random graph on n vertices in which every edge in the complete directed on n 
vertices is included with probability p. Let p = d and let d = w(1). We will call 
this class of graphs G,. Note that if we randomly choose a graph from Œp, the 
underlying undirected graph is randomly chosen from Gp. 


Theorem 1. For sufficiently large n, d = w(1), and p = 4, if we randomly 
choose a graph G € Gp, then with high probability, the ratio PR(G )/P(G) < 
1.64. 


Let E represent the edges in the complete undirected graph Kn for some fixed 
n. Let A C E represent the edges in an undirected graph G chosen at random 
from Gp. Let €; be a small positive constant whose value can be arbitrarily small 
for sufficiently large n. We weight the edges in E as follows: 


n 
j=- if eA, 
mtd (1 = €1)2d : ee ó 
wij =1 if ije E-—A. 
We will refer to this weighted graph as G”. 
Lemma 4. The minimum 7y-bisection of G’ has negative value with high proba- 
bility. 


Proof. By Lemma 3 with high probability the minimum y-bisection of G has at 
least (1 — €,)y(1 — y)2nd edges. Thus, with high probability the total weight of 
the edges in the minimum 4-bisection of G’ is at most: 


n 


(1 = J 
y(i — 7) (r — (1 — e1 )2nd + (1 — e1)2nd(--—"—) = 
y(1 — y) (—(1 — €1)2nd) < 0. 


(1 = y)n? — (1 — a )y(1 — y)2nd + (1 — e1)y(1 — y)2nd(— 
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Lemma 5. Let {vi}, i € V be a set of unit vectors that satisfy the following 
constraints: 


a Vi Uj = 0 (14) 


i jEV 


1+ 0; "Vj 
D A 2 (1-e)Al: (15) 
ijEA 


If e2 < .36, then we can find a y-bisection of G' with a strictly positive value. 
To prove Lemma 5, we will use the following theorem from [GW95]. 


Theorem 2.7 [GW] Let W_ = $, -; w;;, where x7 = min(0, x). Then 


i<j tj? 


{E[W] -W_} > a = Sy wyll — v j) = W- 


i<j 


Proof of Lemma 5: We will use Goemans-Williamson’s random hyperplane 
algorithm to show that we can find a cut that is roughly balanced and has a 
strictly positive value. Let W represent the total weight of the edges that cross 
the cut obtained from a random hyperplane. Let W_ denote the sum of the 
negative edges weights, i.e. W_ = A. Applying Theorem 2.7 from [GW], we 
have: 


1 
E(W| >a 5 2 wall- v: vj) — W- +W_ 


i<j 
l — vi- vj l+ viv; 
j a J 
>a D w m eng] REE. 
i<j:wij>0 i<j:wij <0 


l—v; 


i<jiwij>0 2 +. By condition (14), we have 


EE 2 
that 5°; jey vi: vj =0 and therefore >? IUU = naa, 


We want to calculate the value of $, 


i<j 
F= uiv; 1 — v;i- uj 1 — vi: vj 
2 9 E 2 9 = oy 9 
i<j wiz >O i<j i<j:Wij <0 
l—v-v; nd Uj * Uj 
7 a 23 2t 2 2 
i<j i<jiwij <0 
l1— v; v; nd (1—2ez)nd 
> ae, a ee 
eg ge 
t<7 
_ n? — 2n 


m €ond. 





4 
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Now we have: 


n? —2n n n 


A pe eond) + Geza)! ai end) = aoo 





BW) af 


For large enough n, we can choose €, to be arbitrarily small. So E[W] can be 
bounded from below by a value arbitrarily close to the following: 


(Š a m are ag ~in? = oln?) > (.1585 — =n? _ o(n?). (16) 


If the value of €2 is such that the quantity on line (16) is strictly greater than Bn? 
for some positive constant @, then we will have a contradiction for sufficiently 
large n. Note that if this value is at least @n?, then each side of the cut contains 
at least Øn vertices, so it is a /B-bisection. This value will be strictly positive 
as long as €z < .36. Thus, it must be the case that e2 > .36. O 


Proof of Theorem 1: We fix positive constants y,€;. Suppose we choose a 
random directed graph G as prescribed and let the graph G = (V, A) be the 
undirected graph corresponding to the underlying undirected graph of G. We 
then weight the edges in the graph Kn as discussed previously and obtain G”. 
By Lemma 4, the minimum 4-bisection of G’ is negative with high probability. 
Thus, with high probability equation (15) hold only when e3 > .36. 

Suppose the maximum acyclic subgraph of G, i.e. P(G) is (4 + 9)|A| for 
some positive constant ô. Then the value of Pr(G) is upper bounded by the 
maximum value for some set of unit vectors {v;} of (10), (11), and (12) summed 
over all edges in A. Note that this is equivalent to the quantity in (13) (which 
is no more than .64|A|) plus the quantity in (4). By Lemma 1, the difference 
between (4) and (5) must be no more than 26|A|. Thus, we can upper bound 
the value of Pr(G) by .64|A] + (26 + $(.36 — 26))|A] = (.82 + 6)|A]. Thus, with 
high probability, we have: 

PrR(G) _ .82+6 _ .82 


< g elo 
Aa soa h 








5 Discussion 


In this paper, we make a connection between cuts and vertex ordering of graphs 
in order to obtain a new semidefinite programming relaxation for the linear 
ordering problem. We show that the relaxation is “good” on random graphs 
chosen with uniform edge probability, 1.e. the integrality gap is strictly less than 
2 for most of these graphs. We note that we can extend this theorem to show 
that this relaxation is “good” on graphs that have no small y-bisections for some 
constant y > 0. 

In [HZO1], Halperin and Zwick give a .8118-approximation for a related prob- 
lem that they call the max %-directed-uncut problem. Given a directed graph, 
the goal of this problem is to find a bisection of the vertices that maximizes the 
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weight of the edges that cross the cut in the forward direction plus the weight of 
the edges that do not cross the cut. We note that a weaker version of Theorem 1 
follows from their .8118-approximation algorithm. This is because their semidef- 
inite program for the max %-directed uncut problem is the sum over all edges 
of terms (10), (11), and (12). If for some directed graph G = (V, A), Pr(G) has 
value at least (1 — €)|Al, then the value of their semidefinite programmming re- 
laxation also has at least this value. Thus, if € is arbitrarily small, we can obtain 
a directed uncut of value close to .8118 of the edges, which is a contradiction for 
a random graph with uniform edge probability. In this paper, our goal was to 
give a Self-contained proof of this theorem. 

We would like to comment on the similarity of this work to the work of Pol- 
jak and Delorme [DP93] and Poljak and Rendl [PR95] on the maxcut problem. 
Poljak showed that the class of random graphs with uniform edge probability 
could be used to demonstrate an integrality gap of 2 for several well-studied 
polyhedral relaxations for the maxcut problem [Pol92]. These same graphs can 
be used to demonstrate an integrality gap of 2 for several widely-studied polyhe- 
dral relaxations for the linear ordering problem [NVO1]. The similarity of these 
results stems from the fact that the polyhedral relaxations for the maxcut prob- 
lem are based on odd-cycle inequalities and the polyhedral relaxations for the 
linear ordering problem are based on cycle inequalities. Poljak and Delorme 
subsequently studied an eigenvalue bound for the maxcut problem that is equiv- 
alent to the bound provided by the semidefinite programming relaxation used 
in the Goemans-Williamson algorithm [GW95]. Despite the fact that random 
graphs with uniform edge probability exhibit worst-case behaviour for several 
polyhedral relaxations for the maxcut problem, Delorme and Poljak [DP93] and 
Poljak and Rendl [PR95] experimentally showed that the eigenvalue bound pro- 
vides a “good” bound on the value of the maxcut for these graphs. This exper- 
imental evidence was the basis for their conjecture that the 5-cycle exhibited 
a worst-case integrality gap of 0.88445... for the maxcut semidefinite relax- 
ation [DP93,Pol92]. The gap demonstrated for the 5-cycle turned out to be very 
close to the true integrality gap of .87856... [FS]. 
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Abstract. We propose the MIN-MAX MULTIWAY CUT problem, a variant 
of the traditional MULTIWAY CUT problem, but with the goal of mini- 
mizing the maximum capacity (rather than the sum or average capacity) 
leaving a part of the partition. The problem is motivated by data parti- 
tioning in Peer-to-Peer networks. The min-max objective function forces 
the solution not to overload any given terminal, and hence may lead to 
better solution quality. 

We prove that the MIN-MAX MULTIWAY CUT is NP-hard even on trees, 
or with only a constant number of terminals. Our main result is an 
O(log? n)-approximation algorithm for general graphs, and an O(log? n)- 
approximation for graphs excluding any fixed graph as a minor (e.g., 
planar graphs). We also give a (2 + €)-approximation algorithm for the 
special case of graphs with bounded treewidth. 


1 Introduction 


The MIN-MAX MULTIWAY CUT problem is defined by an undirected graph G = 
(V, E) with edge capacities c(e) > 0, and a set X = {z1,..., £k} C V of distin- 
guished nodes called terminals. A multiway cut is a partition of V into disjoint 
sets S1,...,5% (U; 5; = V}, so that forall i € {1,...,k}, x; € Si. For a partition 
we will use 6(.5;) to denote the capacity of the cut separating S; from the other 
sets LJ jæi 9j, and the goal of the min-max multiway cut problem is to minimize 
the maximum capacity max; 6(5;). 

The min-max multiway cut problem models the data placement problem 
in a distributed database system or a Peer-to-Peer system. In a Peer-to-Peer 
database, the information is stored on many servers. When a user query is issued, 
it is directed to the appropriate server. A request for some data item v can 
lead to further requests for other data. One important issue in such Peer-to- 
Peer databases 1s to find a good distribution of data that minimizes requests to 
any single server. We model this by a graph in which the non-terminal nodes 
represent the data items and the terminals represent the servers. Nodes in the 
partition S; correspond to the data that will be stored on server z. Edges in the 
graph correspond to the expected communication patterns, i.e., the edge (2;, v) 
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represents the number of queries that users at server 7 issue for the data v, and 
the edge (v, w) represents the expected number of times that a request for data 
v will result in an induced request for data w. Communication costs are incurred 
when a query from one server is sent to another. The goal then is to distribute 
the data among the servers so as to minimize the communication cost incurred 
by any one of them. 

The min-max multiway cut problem is closely related to the traditional mul- 
tiway cut problem of [2]. The difference is in the objective function. Unlike the 
min-max multiway cut, in which we seek to minimize the maximum capacity 
max; 6(5;), the multiway cut problem evaluates a partition by the sum of the 
capacities of all edges that connect the parts, thus minimizing the average ca- 
pacity 6(S;). Multiway cut has been used to model similar applications of storing 
files on a network, as well as other problems such as partitioning circuit elements 
among different chips [2]. In many situations, however, the min-max objective 
function may be a better representation of the solution quality. Although the 
multiway cut minimizes the average communication cost of the terminals, this 
cost may not be distributed uniformly among them, resulting in a very heavy 
load on some terminals and almost no load on others. The objective of mini- 
mizing the maximum load tries to alleviate this problem by ensuring that no 
terminal is overloaded. 

Multiway cut problem is NP-hard, but there are very good approximation 
algorithms for it [2], [1], [6]. However, they do not translate directly into good 
approximations for min-max multiway cut, because even the optimal solution to 
one problem can be up to a factor of k/2 worse than the optimum for the other. 


Our Results. For two terminals, min-max multiway cut reduces to the well- 
studied minimum s-t cut problem, and hence it can be solved in polynomial 
time. However, as we show, it is already NP-hard for the case of 4 terminals. As a 
result, we focus on designing approximation algorithms. In Section 2, we present 
an O(a - log n)-approximation algorithm for min-max multiway cut, where a = 
log? n for general graphs, and œ = logn for graphs excluding any fixed graph as 
a minor. The algorithm uses an a-approximation algorithm for a new graph cut 
problem, called MAXIMUM SIZE BOUNDED CAPACITY CUT (MaxSBCC). We use 
it as a subroutine in a procedure that resembles the greedy set cover algorithm, 
incurring an additional factor of O(log n) in the approximation guarantee for the 
min-max multiway cut problem. One of the features of our algorithm is that it 
is able to exhibit flexibility when assigning graph nodes to terminals: if the cut 
that is found for one terminal is later discovered to be bad for another terminal, 
then the nodes are reassigned in a way that is good for both. 

We extend our algorithm to a generalization of the problem, in which there 
is a separate bound B; for each terminal x;, and the goal is to find a partition in 
which ô(S;) does not exceed B;. This generalization is useful when the different 
peers corresponding to the terminals have different communication capabilities, 
and can withstand different loads. 

Turning to special cases of min-max multiway cut, we show that it is NP- 
complete even on trees, and develop a (2 + €)-approximation algorithm for trees 
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and graphs with bounded treewidth. What makes the problem hard on trees is 
that an optimal solution does not necessarily assign connected components of the 
tree to each terminal (see Figure 3, in which the black nodes are the terminals, 
and the optimal solution must assign the white node in the middle to one of 
the leaves). As a result, even if we know which edges should be cut, it may be 
hard to determine how to divide the resulting components among the terminals. 
The key idea of our (2 + €) approximation algorithm is to separate the stage of 
finding connected pieces of the graph from the stage of partitioning them among 
the terminals. Then, in the first stage, the problem of finding “good” pieces is 
solved optimally, and in the second stage these pieces are combined to form a 
2-approximate solution. To make the dynamic programming algorithm of the 
first stage run in polynomial time, the edge capacities are rounded, leading to 
an overall (2 + €)-approximation. 


2 Min-Max Multiway Cut in General Graphs 


2.1 Approximation Algorithm 


Our main goal in this section is to provide an approximation algorithm for the 
min-max multiway cut problem and its extension with nonuniform bounds on 
the capacities. 

First we briefly recall the 2-approximation algorithm of Dahlhaus et al. [2], 
as it is useful to understand why it does not work for the min-max version of 
the problem. Assume that there is a multiway cut with maximum capacity at 
most B. The algorithm finds a minimum capacity cut (S;,7;) separating each 
terminal x; from all other terminals. It is not hard to see that the minimum 
cuts with smallest source sides 5; are disjoint. Let Sọ be the nodes not in any 
S;, and let 6; = 6(.S;) be the capacity of the cut (S;,7;). The cut (S;,7;) is of 
minimum capacity, so we must have 6; < B. The algorithm of [2] assigns each 
set S; to terminal x;, and assigns the remaining nodes Sọ to one of the terminals 
(the one with maximum 4;). This yields a 2-approximation (or, more precisely, 
a 2(1 —1/k) approximation) algorithm for the multiway cut problem, but it is 
only a (k —1)-approximation for the min-max multiway cut problem, as the part 
Si U So can have capacity as high as )/,4;5(S;) < (k —1)B. 

The idea of our algorithm is to take cuts around each terminal that are larger 
in size than the minimum cut. Assume that we are given a bound B, and assume 
that there is a multiway cut where each side has capacity at most B. We will 
use a binary search scheme to optimize B. For a given value of B, we will need 
a subroutine for the following maximum size bounded capacity cut problem. 


Definition 1. Given a graph G = (V, E) with two distinguished vertices s and 
t, weights on vertices w(v), capacities on edges c(e), and an integer B, the MAX- 
IMUM SIZE BOUNDED CAPACITY CUT (MaxSBCC) problem is to find an s-t cut 
(S,T) such that 6(S) < B and w(S) = J` „es w(v) is maximized. 


The MaxSBCC problem can be shown to be NP-hard using a reduction from 
KNAPSACK. For a > 1 and a constant 0 < 6 < 1, let us define an (a, 8)- 
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approximation algorithm for MaxSBCC as an algorithm that, given an instance 
of MaxSBCC with a bound B and an (unknown) optimal solution (S*,7%*), 
produces in polynomial time an s-t cut (S’,T’), such that 6(S’) < aB and 
w(S') > Bw(S*). 

First we show how to use any such approximation algorithm as a subroutine 
for solving the min-max multiway cut problem, and later we give a specific 
(log?n, 1) algorithm. The idea is analogous to the greedy log n-approximation for 
the set-cover problem. Starting from the set V of unassigned nodes of the graph, 
our algorithm iteratively finds (approximate) maximum size bounded capacity 
cuts around each terminal, and temporarily assigns nodes to terminals, until no 
unassigned nodes remain. One important difference is that our algorithm is not 
greedy, in the sense that assignment made to terminals in one iteration can be 
revised in later iterations if that becomes useful. The full algorithm is shown in 
Figure 1. 


1. Initialize S; = {xi} for i =1,...,4, and initialize the weights w(wv) for all v € V by 
setting w(x;) = 0 for all i, and w(v) = 1 for all other nodes. 
2. While U; S fy 
— For each terminal x; € X, 
(a) Construct a graph G” labeling x; as source s and contracting all other 
terminals into a single sink t. 
(b) Find an (a, 8)-approximate MaxSBCC (S,T) in graph G" with bound B 
and weights w(v). Note that the set S does not have to contain S; and 
does not have to be disjoint from the other sets S; for 7 + i. 
(c) Consider the intersection J; = SS; for each 7 Æ i. We need to delete 
this intersection either from 5; or from S. If e(J;, $; \ G) < e(;,5 \ J;), 
then let S$; = 5S; \ J;; otherwise let S= § \ J;. 
(d) Let S: = S: U S, and set the weights of all v € S to w(v) = 0. 
3. Return $1,105 Sk. 


Fig. 1. Min-max multiway cut algorithm 


Theorem 1. /f there is an (a, 3)-approzimation algorithm for MaxSBCC, then 
the above algorithm is an O(@log;,g)-approrimation for MIN-MAX MUL- 
TIWAY CUT problem. 


The key to the analysis is to see that each iteration assigns a constant fraction 
of the remaining nodes. By assumption there is a multiway cut (Sf,..., 5%) with 
capacity B. For each terminal z;, we use the approximation bound to claim that 
the application of the MaxSBCC assigned at least as many new nodes to x; as 
a 8 fraction of the remaining nodes in S;. 


Lemma 1. Jf there is a multiway cut with maximum capacity at most B, then 
in any iteration of the while loop, if U is the set of unassigned nodes in the 
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beginning of the iteration, and U’ is the set of unassigned nodes at the end of 
this iteration, then \U"| < 43|UI- 


Proof. Let N; be the set of previously unassigned nodes added to the set S; in 
this iteration. Notice that step (2c) of the algorithm only reassigns nodes with 
zero weight, so N; has the same weight as the solution to MaxSBCC S obtained 
in step (2b). 

Consider some optimal solution (Sf,...,S%) to the min-max multiway cut 
instance. Now partition U” into sets Uj,...,U;, such that Uf = S¥ MU’. We claim 
that w(N;) > 6-|U;|. To see this, notice that the nodes in U’ have weight 1 
throughout this iteration of the while loop, and since S¥ is a piece of the optimal 
partition, 6(S*) < B. Therefore, in the it? iteration of the for loop, (Sž, V \ St) 
is a feasible solution to the MaxSBCC problem, and w(S?) > w(Uj) = |Ujj. 
By the (a, 2)-approximation guarantee, the algorithm for MaxSBCC must find 
a set with w(N;) > 2- |Uj|. Summing over all i, we obtain that |U| — |U’| = 
we w(N;) > 8- |U'|, which proves the claim. C 


Proof (of Theorem 1). By using binary search, we can assume that a bound B 
is given, and our algorithm will either prove that no multiway cut of maximum 
capacity at most B exists, or it will find a multiway cut with capacity at most 
O(alog;,,n)B. 

Throughout the algorithm, x; € S; for all 7, and the sets S; are always 
disjoint. So the algorithm finds a multiway cut, as required. By Lemma 1 the 
algorithm terminates in at most log,,, n iterations of the while loop, if a min- 
max multiway cut of capacity at most B exists. If given an infeasible bound 
B < B*, it may not stop after log,,,n iterations, which proves that B < B*. 
This shows that the algorithm runs in polynomial time. We will also use this 
bound to give an approximation guarantee for the algorithm. 

We claim that for each S; returned by the algorithm, 6(S;) < alog,,,n-B. 
To see this, notice that for each application of the MaxSBCC subroutine in (2b), 
the capacity of the set S returned is at most 6(S) < aB. By the choice made 
in step (2c), the transfer operation does not increase either 6(S;) or 6(S). Soin 
each iteration of the while loop, the capacity of each S; increases by at most 
aB. Combined with the bound on the number of iterations, this observation 
concludes the proof. o 


Feige and Krauthgamer [3] give an O(log? n) approximation algorithm for the 
problem of finding cuts with specified number of nodes, and an improved O(log n) 
approximation for the case when the input graph G is assumed not to contain 
a fixed graph as a minor (e.g., for planar graphs). We will use this algorithm 
to give an (O(log? n),1) approximation algorithm for the MaxSBCC problem in 
general graphs and an improved (O(log n),1) approximation algorithm in the 
special case. This will yield the following theorem. 


Theorem 2. There is an O(log? n)-approrimation algorithm for MIN-MAX 
MULTIWAY CUT problem on general graphs, and an O(log? n)-approximation for 
graphs excluding any fixed graph as a minor (e.g., planar graphs). 
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Proof. Feige and Krauthgamer [3] give an algorithm for finding cuts with spec- 
ified sizes. For a graph G with n nodes and each number d < n, their algorithm 
finds a cut (Sa, Ta) with |Sg| = d and capacity (Sa) within a = O(log? n) of 
the minimum capacity for such a cut. For graphs excluding any fixed graph as 
a minor, this guarantee is improved to a’ = O(log n). The algorithm also works 
for finding s-t cuts on graphs with node weights and edge capacities. 

We claim that the cut that corresponds to the largest value d* such that 
6(Sqg+) < aB is an (a,1)-approximate MaxSBCC. By definition, its capacity 
is at most aB. And, if the optimal MaxSBCC had size d’ > d*, then, by the 
guarantee of the algorithm, ô( Sa) would be at most aB, contradicting our choice 
of d*. The result then follows from Theorem 1. o 


It is interesting to note that the algorithm can also be used for a version of 
the multiway cut problem in which there is a separate bound B; for each ô(S;). 
To obtain the extension, we use the MaxSBCC algorithm in each iteration 2 of 
the for loop with bound B; rather than B. 


Theorem 3. Assume we are given a graph G with k terminals, edge capacities, 
and k bounds (Bı,..., Bk). If there is a multiway cut (S;,...,S,%) such that 
6(S;) < Bi for each i, then in polynomial time we can find a multiway cut 
(S{,...,5%) such that 6(S!) < O(log? n)B;, and the bound improves by a factor 
of logn for graphs excluding any fixed graph as a minor. 


Remark. Calinescu, Karloff and Rabani [1] and subsequently Karger et al. [6] 
gave improved approximation algorithms for the multiway cut problem based on 
linear programming and rounding. It appears that this technique does not yield 
a good approximation for our problem. To see this, consider the graph which is 
a star with k terminals and a single additional node at the center, and assume 
the capacity of each edge is 1. There is no multiway cut where each part has 
capacity at most B = 2, or even approximately 2. By assigning the center of the 
star to terminal x;, we can get a multiway cut where the capacity of each part S; 
for 7 Æ iis 1, while the capacity of Siis k — 1. A linear programming relaxation 
would allow us to take a “linear combination” of these cuts, and thereby have 
each side have capacity at most 2. 


2.2 NP-Completeness 


We prove using a reduction from BISECTION that the MIN-MAX MULTIWAY CUT 
problem is NP-hard already on graphs with 4 terminals. 


Theorem 4. MIN-MAX MULTIWAY CUT is NP-hard for any fixed k > 4 even 
with unit-capacity edges. 


Proof. We will show that it is NP-hard for k = 4 using a reduction from BISEC- 
TION [4]. Our construction uses capacities, but we can replace each edge with 
multiple parallel paths. An instance of BISECTION consists of a graph G = (V, E) 
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Fig. 2. Reduction from BISECTION to MIN-MAX MULTIWAY CUT 


with an even number of vertices n, and an integer C. The question is whether 
or not there exists a partition of V into two sets X and Y, each of size n/2, 
such that the capacity of the cut e( X,Y} < C. Given G and C, we construct, in 
polynomial time, a graph F with 4 terminals and a bound B, so that F has a 
multiway cut with maximum capacity at most B if and only if G has a bisection 
with capacity at most C. 

We obtain the graph F = (V’, E’) by adding 4 new terminal nodes X = 
{u,d,l,r} to G, and adding edges that connect nodes of G to the terminals (see 
Figure 2). E’ includes E and the following additional edges, where a is chosen 
such that 2a > C: 


— Edge (u, d) of capacity na 
— Edges (v,u) and (v, d), each of capacity a, for each v € V. 
— Edges (v,/) and (v,r), each of capacity b = a — s > 0, for each v € V. 


The bound is set to B = 2na. 

Suppose F has a min-max multiway cut (U U {u}, DU {d}, LU {1}, RU {r}) 
where each part has capacity at most B. Then U and D must be empty, as 
B = 2na > 6(U U {u}) > 2na + 2b|U], just counting the edges to the terminals. 
So (L, R) is a cut of G, and let C’ = c(L,R) denote its capacity. The next 
observation is that |L| = |R| = n/2. To see this, suppose, for contradiction, that 
|L| =k > 3 +1 (or similarly for |R|). Then 


C 
§(LU {I}) = 2ka + nb +O! > (F + 1)a + n(a — —) = 2na + (2a — C) > B, 


where the last inequality follows from the choice of a. We conclude that the 
capacity C” of the bisection (L, R) must be at most C. This follows as the 
capacity of the cut DU {l} is na+nb+C’ < B = 2na, and by the choice 
of b this inequality implies C’ < C. To show the opposite direction, given a 
bisection (X, Y) in G of capacity C” < C, we produce a min-max multiway cut 
({u}, {d}, XU{l}, YU{r}) of F, with each component’s capacity at most B. O 
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3 Min-Max Multiway Cut on Trees 
and Bounded Treewidth Graphs 


Recall from the Introduction that in an optimal solution to the min-max multi- 
way cut problem on trees the sets of nodes assigned to the terminals do not have 
to be connected. This can be seen in the example of Figure 3. All nodes except 
for the middle one are terminals, and all edges have capacity 1. The optimal 
solution cuts all the edges incident on the middle node and assigns it to one of 
the leaf (degree-one) terminals, achieving a value of 4. On the other hand, any 
solution that assigns connected parts of the graph to each terminal would leave 
the middle node connected to one of its neighbors, incurring a cost of 5. 


Fig. 3. Example showing that in an optimal min-max multiway cut on a tree, the sets 
assigned to the terminals need not form connected components. The only non-terminal 
in this graph is the middle node 


In Section 3.1 we use this observation to prove that the MIN-MAX MULTIWAY 
CUT problem in NP-hard on trees. Then we provide a (2 + €)-approximation on 
trees, and, finally, in Section 3.4 we extend it to graphs with bounded treewidth. 


3.1 NP-Completeness 


Theorem 5. MIN-MAX MULTIWAY CUT is strongly NP-hard when the graph is 
a tree with weighted edges. 


Proof. We use a reduction from 3-PARTITION, which is known to be strongly 
NP-complete [5]. In 3-PARTITION, given a set A = {a@1,...,@3m}, a weight w; for 
each a; € A, and a bound B, such that Vi B/4 < w; < B/2 and D wi = mB, 
we want to know if A can be partitioned into disjoint sets $1, ..., Sm, such that 
foreach 7, 


ai€S; 


Given an instance (A, B) of 3-partition, we construct an instance of min-max 
multiway cut as follows. The tree T consists of separate subtrees connected with 
zero-capacity edges. There will be 3m subtrees T;, one for each element a;, and 
m isolated terminals x1, ..., £m, one for each of the desired sets. Each T; consists 
of six terminals and one non-terminal v;, with edge capacities as in Figure 4. 
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B-w;/3 


Fig. 4. Component T; used in the NP-completeness reduction for MIN-MAX MULTIWAY 
CUT on trees 


We claim that a min-max multiway cut of maximum capacity at most B 
exists if and only if the 3-partition instance is solvable. Notice that any min- 
max multiway cut with capacity at most B must cut all edges of T; and assign 
all v;’s to the terminals z1, ..., £m, creating a partition of A. If a set of nodes S; 
is assigned to terminal xj, then the capacity of the resulting part is }/,,¢ s, Wi. 


This implies that such a cut exists 1f and only 1f the 3-partition does. O 


3.2 Algorithm for Min-Max Multiway Cut on Trees 


In this section we give a (2 + €)-approximation algorithm for the min-max mul- 
tiway cut on trees that have edge capacities. 

The algorithm consists of two stages. In the first stage we consider a variant 
of the problem where we allow the algorithm to create extra parts in the partition 
that do not contain terminals. More precisely, we consider the following problem. 


Definition 2. The TREE CUTTING problem (T,X,B) for a tree T = (V,E), 
terminals X = {2},...,2~%} C V, and a bound B is to find a partition of V 
into connected subtrees Ti, ..., Th, subject to the following constraints: (1) no 
two terminals are in the same connected component; and (2) for each connected 
component T;, 6(T;) < B. The objective is to minimize $; 6(T;). 


In the next subsection we give a pseudo-polynomial time algorithm for this 
problem. Here we show how to use such an algorithm to get a (2 + €)-approxi- 
mation for the min-max multiway cut on trees. 


Theorem 6. Using a pseudo-polynomial time algorithm for the TREE CUTTING 
problem as a subroutine, we can give a polynomial time (2 + €)-approzimation 
for the MIN-MAX MULTIWAY CUT on trees. 


Proof. First we give a 2-approximation for min-max multiway cut that uses a 
pseudo-polynomial exact algorithm for tree cutting. Given a tree T with termi- 
nals X, we will binary search for a lower bound B* to the min-max multiway cut 
optimum. Observe that connected components in a feasible solution to min-max 
multiway cut instance (T, X) of value B give a feasible solution to the tree cut- 
ting instance (T, X, B) of value at most kB. Therefore, if our optimal tree cutting 
solution T}, ..., Tp does not satisfy D 6(T;) < kB, then B < B*. The algo- 
rithm groups the components T; into k sets §1,..., Sk of nodes greedily, assigning 
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the terminals to different sets. Observe that if several components, say T}, ..., T}, 
are combined into a set S, then 6(S) < $2; 6(T;). Because we 6(T;) < kB, 
and each 6(S;) is at least B, all components will be packed into the k sets. Also, 
since for all j, 6(7;) < B, for no ż will 6(S;) exceed 2B. 

Recall that our tree cutting algorithm runs in pseudopolynomial time. We 
obtain a polynomial-time (2 + €)-approximation algorithm via rounding. For a 
given €, capacity bound B, and m = |E], let 


[m/e 
a 


For each edge e € E, scale the capacity so that c'(e) = |ac(e)|. Also set B’ = 
aB = [m/e]. If there is a multiway cut with maximum capacity B in the original 
problem, then there is one of maximum capacity at most B’ after rounding. Now 
we obtain a 2-approximate multiway cut S4, ..., S% for the graph with capacities 
c'(e) and bound B’. The running time is polynomial in B’ = [m/e] and n. The 
capacity of a part S; of this partition is at most 6(S;) < (2 +€) - B using the 
original capacities. O 


3.3 Algorithm for the Tree Cutting Problem 


We now describe an algorithm that solves optimally, in time polynomial in B and 
the size of the tree n, the tree cutting problem (which we used as a subroutine 
for the min-max multiway cut on trees). To simplify the presentation of the 
algorithm, assume, without loss of generality, that (1) T is rooted at a node r 
and all edges are directed away from the root; (2) T is binary. (To make the tree 
binary without affecting the solution, replace each node u that has d > 2 children 
with a [loga d—1]-height complete binary subtree U with edge capacities B + 1, 
and attach u’s children to the leaves of U, at most 2 per leaf.) 

The tree cutting problem will be solved using dynamic programming. First 
consider the simpler problem with no terminals. To solve this problem, we con- 
struct a dynamic programming table p(v, A) for all nodes v € V and integers 
0 < A < B, where the entry p(v, A) is the minimum total capacity of edges in 
the subtree of T rooted at v that can be cut such that the total capacity of edges 
coming out (i.e., toward descendants) of v’s component is at most A. We have 
the separate bound A because the remaining B — A capacity will be used to cut 
the edges that are incident on v’s component, but lie outside of its subtree. The 
values p(v, A) can be computed in a single pass up the tree. If a node v has one 
child vı, then cutting (v,v1) implies that the component containing vı can have 
at most B — e(v, vı) capacity below v1, so the total capacity obtained this way 
is c(v, v1) + p(v,, B — e(v,v1)). If we do not cut the edge (v,v,), then we get 
p(v1, A). This leads to the following recurrence for the case that v has a single 
child vı. 


p(vi, A) if c(v, v) > A 
min{p(v;, A), e(v, v1) + p(v1, B — e(v, v1))} otherwise. 


peA = { 
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Now suppose that the internal node v has two children, vı and v2. Then the 
capacity A available for cutting edges below v has to be partitioned between the 
edges that belong to the left subtree (including, possibly, (v, v1)), and the ones 
that belong to the right subtree (possibly including (v,v2)). The algorithm tries 
all possibilities for such a partition Aı + Ag = A. Then, given 4;, it decides 
independently for each child node v; whether or not to cut (v,v;), using an 
expression similar to the one above. 

Next we extend the algorithm to make sure that all terminals in the tree are 
separated. For this, a binary variable t is added to the parameters of the table. 
The value of ¢ limits the options available to the above simpler algorithm in each 
step. It will either require that a given component not contain a terminal (t=0), 
or it will not impose such a restriction (t = *). The idea is that if a node is 
connected to some ancestor which is a terminal, then it may not be connected 
to any descendants which are terminals, so in this case we will use a table entry 
with t¢ = 0. Also, care has to be taken that a node is not connected to two 
terminals which are both descendants. 


Theorem 7. The optimal solution to the TREE CUTTING problem can be com- 
puted in time polynomial in the size of the graph and the bound B. 


3.4 Bounded Treewidth Graphs 


Finally, we extend the (2 + €)-approximation algorithm for min-max multiway 
cut to work on graphs with bounded treewidth (see [7] for an introduction to 
tree decomposition). The only change is in solving the tree cutting problem. 

First we note that the algorithm from Section 3.3 can work on trees with 
degree greater than two. The only potential difficulty is to generate the optimal 
guesses of Aj,..., Aa, >, A; = A, in polynomial time. Given the values of the 
subproblems at the leaves, the problem of finding the best partition to obtain the 
value p(w, A) is essentially a knapsack problem, and hence is solved by optimally 
solving the problem for eachsuffix j,...,d of the set of node’s children, with j 
going from d to 1, and using the optimal result for 7,...,d in order to solve the 
problem for j — 1,...,d. 

Now we sketch how to extend this algorithm to handle graphs with bounded 
treewidth. Suppose that we are given a graph G = (V, E) and a decomposition 
tree T for it, such that for each node u in T, there is a set Va C V associated with 
it. The size of each V, is bounded by a constant b. Let us root T at some node r 
and assign a height h(u) to each node u € T so that r is the highest. Now we can 
associate with each vertex v € V a label, which is defined as max{h(u)|v € Va}. 
The algorithm will again build a dynamic programming table. In this case a 
table entry will be p(u, {H1, ..., Hn}, {41, ..., An}, {t1; -th }), where u is a node 
in T; H;’s form a partition of V, that has h components, for some h € [1,..., b], 
with the meaning that different H;’s will be subsets of different components in 
the solution; A; is a bound on the total capacity of edges (v,w) such that v 
is in the same component as Hj, w is not, and label(v) > label(w); and t; is 
a variable for H; that specifies, as before, whether this component is allowed 
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to contain terminals. The computation proceeds in a bottom-up fashion on the 
tree T. When a node u of T and its child node wu, are considered, all allowed 
combinations of vertices in V, and V,,, are evaluated, subject to the constraints 
imposed by H;’s and t;’s, and the best one is chosen. If u has multiple children, 
then each A; is divided among the children in the same way as before. The 
running time of the algorithm is exponential in b, but, given that b is a fixed 
constant, it remains polynomial (or pseudopolynomial, as before). 


Theorem 8. The above algorithm computes the optimal solution to the TREE 
CUTTING problem in graphs with bounded treewidth in time polynomial in the size 
of the graph and the bound B. As a consequence, we get a (2 + €)-approximation 
for the MIN-MAX MULTIWAY CUT in graphs with bounded treewidth. 
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Abstract. Given any integer d > 3, let k be the smallest integer such 
that d < 2klogk. We prove that with high probability the chromatic 
number of a random d-regular graph is k, k+ 1, ork +2. 


1 Introduction 


In [10], Luczak proved that for every real d > 0 there exists an integer k = k(d) 
such that w.h.p.' x(G(n,d/n)) is either k or k +1. Recently, these two possible 
values were determined by the first author and Naor [4]. 

Significantly less is known for random d-regular graphs Gna. In [6], Frieze 
and Luczak extended the results of [9] for x(G(m, p)) to random d-regular graphs, 
proving that for all integers d > do, w.h.p. 


d d log log d 
hee ee ee 
Gna) = ioga =O (Tapa) 
Here we determine x(Gn,a) up to three possible values for all integers. More- 


over, for roughly half of all integers we determine x(Gn a) up to two possible 
values. We first replicate the argument in [10] to prove 


Theorem 1. For every integer d, there exists an integer k = k(d) such that 
w.h.p. the chromatic number of Gn a is either k ork +1. 


We then use the second moment method to prove the following. 


Theorem 2. For every integer d, w.h.p. X(Gn a) is either k, k+1, ork+2, where 
k is the smallest integer such that d < 2klogk. If, furthermore, d > (2k—1) log k, 
then w.h.p. X(Gn a) is either k +1 ork +2. 


The table below gives the possible values of x(Gn,a) for some values of d. 









6 7,8,9 10 100 1,000,000 
45 45,6 5,6 18,19,20 46523, 46524 










x(Gn.d) 3,4 3,4,5 


' Given a sequence of events En, we say that £ holds with positive probability (w.p.p.) if 
lim infp_,/oo Pr[En] > 0, and with high probability (w.h.p.) if lim infpoo Pr[En] = 1. 


K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 219-228, 2004. 
© Springer-Verlag Berlin Heidelberg 2004 
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1.1 Preliminaries and Outline of the Proof 


Rather than proving our results for Gn, a directly, it will be convenient to work 
with random d-regular multigraphs, in the sense of the configuration model [5]; 
that is, multigraphs C,4 generated by selecting a uniformly random configura- 
tion (matching) on dn “vertex copies.” It is well-known that for any fixed integer 
d, a random such multigraph is simple w.p.p. As a result, to prove Theorem 1 
we simply establish its assertion for Cy g. 

To prove Theorem 2 we use the second moment method to show 


Theorem 3. Ifd < 2klogk, then w.p.p. X(Cna) < k +1. 


Proof of Theorem 2. For integer k let up = (2k — 1)logk and c = 2klogk. 
Observe that cy < uk < cy. Thus, if k is the smallest integer such that d < cp, 
then either 1) uk < d < Ck OT 11) Uk—1 < Ck-1 < d < UR < Cy. 

A simple first moment argument (see e.g. [11]) implies that if d > ux then 
w.h.p. x(Caa) > k. Thus, if ug < d < cy, then w.h.p. Cna is non-k-colorable 
while w.p.p. it is (k + 1)-colorable. Therefore, by Theorem 1, w.h.p. the chro- 
matic number of C, 4g (and therefore Gn,a) is either k +1 or k +2. In the second 
case, we cannot eliminate the possibility that Gn, a is w.p.p. k-colorable, but we 
do know that it is w.h.p. non-(k — 1)-colorable. Thus, similarly, it follows that 
X(Gn a) is w.h.p. k, k+1 or k +2. o 


Throughout the rest of the paper, unless we explicitly say otherwise, we 
are referring to random multigraphs Cn, a. We will say that a multigraph is k- 
colorable iff the underlying simple graph is k-colorable. Also, we will refer to 
multigraphs and configurations interchangeably using whichever form is most 
convenient. 


2 2-Point Concentration 


In [10], Luczak in fact established two-point concentration for x(G(n,d/n)) for 
all € > 0 and d = O(n1/®-£). Here, mimicking his proof, we establish two-point 
concentration for X(n a) for all € > 0 and d = O(n!/-6). 

Our main technical tool is the following martingale-based concentration in- 
equality for random variables defined on Cn,a [12, Thm 2.19]. Given a configu- 
ration C, we define a switching in C to be the replacement of two pairs {€1, eg}, 
{e3,e4} by {e1,e3}, {e2,e4} or {e1, ea}, {e3, e2}. 


Theorem 4. Let X, be a random variable defined on Cn a such that for any 
configurations C, C’ that differ by a switching 


IXn(C) — Xn(C)| <b, 
for some constant b > 0. Then for every t > 0, 


Pr[Xn <E[Xp]—t] <e 7T and Pr[Xn > E[Xn] +t] <e ae 
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Theorem 1 will follow from the following two lemmata. The proofof Lemma 1 
is a straightforward union bound argument and is relegated to the full paper. 


Lemma 1. For any 0 < e < 1/6 and d <n}/®-<, w.h.p. every subgraph induced 
by s < nd™?0+26) vertices contains at most (3/2 — e)s edges. 


Lemma 2. For a given function w(n), let k = k(w,n,p) be the smallest k such 
that 
Prix(Cn a) < k] > 1/w(n) . 


With probability greater than 1 — 1/w(n), all but 8,/ndlogw(n) vertices of Cn ad 


can be properly colored using k colors. 


Proof. For a multigraph G, let Yp(G) be the minimal size of a set of vertices S 
for which G — S is k-colorable. Clearly, for any k and G, switching two edges of 
G can affect Y,(G) by at most 4, as a vertex cannot contribute more than itself 
to Y,(G). Thus, if uz = EfY;(Cn,a)], Theorem 4 implies 


2 


Pr[Yk < uk — Ayn] < e~ fa and =PrlY¥, > uk +AV/n] < eter, (1) 


Define now u = u(n,p,w(n)) to be the least integer for which Pr[x(G) < u] > 
1/w(n). Choosing A = A(n) so as to satisfy e7>*/064) = 1/w(n), the first in- 
equality in (1) yields 


Pr[Yu < pu — AV] < 1/w(n) < Pr[x(G) < u] = Pr[Yu = 0] . 


Clearly, if Pr[Y, < Hu ~ Ayn] < Pr[Y, = 0] then py < Ayn. Thus, the sec- 
ond inequality in (1) implies Pr[Y > 2A/n] < 1/w(n) and, by our choice, 
A = 4,/dlogw(n). E 


Proof of Theorem 1. The result is trivial for d = 1,2. Given d > 3, let 
k = k(d,n) > 3 be the smallest integer for which the probability that Chia 
is k-colorable is at least 1/loglogn. By Lemma 2, w.h.p. there exists a set of 
vertices S such that all vertices outside S can be colored using k colors and |S} < 
8./ndlogloglogn < Vndlogn = so. From S, we will construct an increasing 
sequence of sets of vertices {U;} asfollows. Up = S; for i > 0, UVig1 = U; U 
{w 1, w2}, where w1, wg ¢ U; are adjacent and each of them has some neighbor 
in U;. The construction ends, with Ut, when no such pair exists. 

Observe that the neighborhood of U; in the rest of the graph, N (U+), is always 
an independent set, since otherwise the construction would have gone on. We 
further claim that w.h.p. the graph induced by the vertices in Uz is k-colorable. 
Thus, using an additional color for N (U+) yields a (k + 1)-coloring of the entire 
multigraph, concluding the proof. 

We will prove that U+ is, in fact, 3-colorable by proving that |U;| < so/e. 
This suffices since by Lemma 1 w.h.p. every subgraph H of b or fewer vertices 
has average degree less than 3 and hence contains a vertex v with deg(v) < 2. 
Repeatedly invoking Lemma 1 yields an ordering of the vertices in H such that 
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each vertex is adjacent to no more than 2 of its successors. Thus, we can start 
with the last vertex in the ordering and proceed backwards; there will always be 
at least one available color for the current vertex. To prove |U;| < 2so logn we 
observe that each pair of vertices entering U “brings in” with it at least 3 new 
edges. Therefore, for every 7 > 0, U; has at most so + 27 vertices and at least 37 
edges. Thus, by Lemma 1, w.h.p. t < 3s9/(4e). m 


3 Establishing Colorability in Two Moments 


Let us say that a coloring o is nearly—balanced if its color classes differ in size 
by at most 1, and let X be the number of nearly—balanced k-colorings of Cy a. 
Recall that c = 2klogk. We will prove that for all k > 3 and d < c,_, there 
exist constants Cy, C2 > 0 such that for all sufficiently large n (when dn is even), 


1 dn/2 
B[X] > Cyn- 4 YP pr (1 = z) (2) 


dn 
E[X?] < Cn pan (1 ws =) (3) 


By the Cauchy-Schwartz inequality (see e.g. [7, Remark 3.1]), we have Pr|X > 
0] > E[X]?/E[X2]> C?/C, > 0, and thus Theorem 3. 

To prove (2), (3) we will need to bound certain combinatorial sums up to 
constant factors. To achieve this we will use the following Laplace-type lemma, 
which generalizes a series of lemmas in [2—4]. Its proof is standard but somewhat 
tedious, and is relegated to the full paper. 


Lemma 3. Let £,m be positive integers. Let y E€ Q™, and let M be amx £ 
matrix of rank r with integer entries whose top row consists entirely of 1’s. Let 
s,t be nonnegative integers, and let vi,wj € Né fri <i<sandi<j<t, 
where each v; and w; has at least one nonzero component, and where moreover 
uy v= De w,;. Let f : IR‘ > R be a positive twice-differentiable function. 
For n € N, define 

s 

(v; z)! 

Se — ys Hens (vi-2)! f(z/n)” 

_,(w; +2)! 

(zeNt:M-z=yn} lat J 


and define g : Rf + R as 


Ii- (vi: es 
Tyan (ws 0 


where 0° = 1. Now suppose that, conditioned on M -€ = y, g is maximized at 
some ¢* with ¢¥ > 0 for alli, and write gmax = g{¢*). Furthermore, suppose 
that the matrix of second derivatives g" = 0*g/0¢; O¢; is nonsingular at Ç*. 

Then there exist constants A, B > O, such that for any sufficiently large n 
for which there exist integer solutions Z to M - z = yn, we have 


g(¢) = iG) 
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Sn <B. 


AL aaa a 
n—(é+s—t—r)/2 ge ax 


For simplicity, in the proofs of (2) and (3) below we will assume that n is 
a multiple of k, so that nearly—balanced colorings are in fact exactly balanced, 
with n/k vertices in each color class. The calculations for other values of n differ 
by at most a multiplicative constant. 


4 The First Moment 


Clearly, all (exactly) balanced k-partitions of the n vertices are equally likely to 
be proper k-colorings. Therefore, E[X] is the number of balanced k-partitions, 
n!/(n/k)!*, times the probability that a random d-regular configuration is prop- 
erly colored by a fixed balanced k-partition. 

To estimate this probability we will label the d copies of each vertex, thus 
giving us (dn — 1)!! distinct configurations, and count the number of such config- 
urations that are properly colored by a fixed balanced k-partition. To generate 
such a configuration we first determine the number of edges between each pair 
of color classes. Suppose there are b;; edges between vertices of colors 7 and j 
for each 1 Æ J. Then a properly colored configuration can be generated by 1) 
choosing which b;; of the dn/k copies in each color class ¢ are matched with 
copies in each color class 7 Æ i, and then ii) choosing one of the 5;;! matchings 
for each unordered pair i < 7. Therefore, the total number of properly colored 
configurations 1s 


Leal _ (dn/k)!* 
ER eo = fr 


ll bij! 
jti Dig! ga i<j ii? 


Summing over all choices of the {b;;} that satisfy the constraints 
Vi: X bij = dn/k , (4) 


we get 

n! 1 (dn/k)\* 
(a/R (dn Ay A Ty bs 

n!  (dn/k)!* (dn /2)! 
(E)E (dn)! 2+ TT, bal 


E[X] = 


= gan/2 


By Stirling’s approximation /2mn(n/e)” < n! < V4rn(n/e)” we get 


ae i (dn/2)! 
E[X] > Diam 2 They Bal (5) 


where Di = 27(k+1)/2 q(k-1)/2, 
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To bound the sum in (5) from below we use Lemma 3. Specifically, z consists 
of the variables b;; with i < j, so £ = k(k—1)/2. For k > 3, the k constraints (4) 
are linearly independent, so ae them as M -z = yn gives a matrix M 
of rank k. Moreover, they imply are bi; = dn/2, so adding a row of 1’s to the 
top of M and setting yı = d/2 doce not increase its rank. Integer solutions z 
exist whenever n is a multiple of k and dn is even. We set s = 1 and t = @; the 
vector vı consists of ls and the w; are the £ basis vectors. Finally, f(¢) = 1. 
Thus, £+ s — t — r = —(k — 1) and 


d/2)4/2 1 
(¢) i ( / ) _ — e(4/2)H (2¢/d) 


rr 


a ilhe 


where H is the entropy function H(x) = — Joa x; log Ty. 

Since g is convex it is maximized when ¢¥ = d/(2é) for all 1 < j < £, and g” 
is nonsingular. Thus, gmax = (k(k — 1)/2)¢/? implying that for some A > 0 and 
all sufficiently large n 


gens? klk = 1)\ "P 
ARSD (els a 
E[X] > Digan n X An ( 5 ) 


1 dn/2 
= D, An- -1/2 gn (: E z) 


Setting Cı = D1 A completes the the proof. 


5 The Second Moment 


Recall that X is the sum over all balanced k-partitions of the indicators that 
each partition is a proper coloring. Therefore, E[X?] is the sum over all pairs 
of balanced k-partitions of the probability that both partitions properly color a 
random d-regular configuration. Given a pair of partitions 0,7, let us say that a 
vertex v is in class (i, j) ifo(v) = i and T(v) = j. Also, let a,; denote the number 
of vertices in each class (i, j). We call A = (a;;) the overlap matrix of the pair 
o,T. Note that since both o and 7 are balanced 


Vi: > ay =) aj =n/k (6) 
j 1 


We will show that for any fixed pair of k-partitions, the probability that they 
both properly color a random d-regular configuration depends only on their 
overlap matrix A. Denoting this probability by g(A), since there are n!/ I; , Q;j! 
pairs of partitions giving rise to A, we have 


E[X?] = Jia =A) (7) 


where the sum is over matrices A satisfying (6). 
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Fixing a pair of partitions ø and 7 with overlap matrix A, similarly to the 
first moment, we label the d copies of each vertex thus getting (dn — 1)!! distinct 
configurations. To generate configurations properly colored by both o and 7 we 
first determine the number of edges between each pair of vertex classes. Let 
us say that there are };;,¢ edges connecting vertices in class (7,7) to vertices in 
class (k, £). By definition, bijke = breij, and if both colorings are proper, bijke = 0 
unless i Æ k and 7 Æ £. Since the configuration is d-regular, we also have 


> bijke = daij , (8) 
ki ttj 
To generate a configuration consistent with A and {b;jxe} we now i) choose 
for each class (7,7), which bijke of its daij copies are to be matched with copies 
in each class (k, 2) with k Æ ¿i and £ Æ j, and then ii) choose one of the b;;;,¢! 
matchings for each unordered pair of classes i < k, j Æ £. Thus, 


1 | a 
, a ae seer se ath ŘÁ bijk 
q( ) (dn = 1)!! Be (1 Trz, lLÆŁj bijke! git. £: | 
_ odn/2 Ls (dais)! Cli 


TE T. (9) 
(dn)! baa} Hick, jze bijke! 


where the sum is over the {b;;,¢} satisfying (8). Combining (9) with (7) gives 


(dax)! | 
EIX? = 22 S ni Mylo) (and ag 
{aij} {bijke} Tij ais! pig: (dn)! Lick je bijke! 


To bound the sum in (10) from above we use Lemma 3. We let z consist 
of the combined set of variables {a;;} U {bijne : i < k,j # £l}, in which case 
its dimensionality £ (not to be confused with the color £) is k? + (k(k — 1))?/2. 
We represent the combined system of constraints (6), (8) as M - z = yn. The 
k? constraints (8) are, clearly, linearly independent while the 2k constraints (6) 
have rank 2k —1. Together these imply )7,, aij = 1 and È icp jee bijke = d/2, so 
adding a row of 1’s to the top of M does not change its rank from r = k? +2k—1. 
Integer solutions z exist whenever n is a multiple of k and dn is even. Finally, 
f(O = 29/2, s = k? +2 and t = k? +14 (k(k—1))?/2, so €+s—t—r = —2(k—1). 

Writing œ;; and Bijke for the components of ¢ corresponding to ai;/n and 
bijke/n, respectively, we thus have 


9(¢) = gaa 1 _ Myldag) (a2)? 


Qij d Bij 
Lij ig d Lick 540 Bijke 
a 
1 qa/2 [l;e ij 


ij a EE wea 


In the next section we maximize g(C) over ¢ € R£ satisfying M-¢ = y. We 
note that g” is nonsingular at the maximizer we find below, but we relegate the 
proof of this fact to the full paper. 
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6 A Tight Relaxation 
Maximizing g(C) over ¢ € Rf satisfying M-¢ = y is greatly complicated by the 


constraints 
` Bijke = dois (12) 
ki, bey 

To overcome this issue we i) reformulate g(¢) and ii) relax the constraints, in a 
manner such that the maximum value remains unchanged while the optimization 
becomes much easier. 

The relaxation amounts to replacing the k? constraints (12) with their sum 
divided by 2, 1.e., with the single constraint 


5 Bijke = d/2 . (13) 


i<k,jKe 


But attempting to maximize (11) under this single constraint is, in fact, a bad 
idea since the new maximum is much greater. Instead, we maximize the following 
equivalent form of g(¢) 


| da? I: ee tj Bizke 
Xij Pijki 
I; Aij Loa Pijke 


derived by using (12) to substitute for the exponents da;; in the numerator of 
(11). This turns out to be enough to drive the maximizer back to the subspace 
M (=y. 

Specifically, let us hold {a,;} fixed and maximize g(¢) with respect to {8i;jke} 
using the method of Lagrange multipliers. Since log g is monotonically increasing 
in g, it is convenient to maximize log g instead. If À is the Lagrange multiplier 
corresponding to the constraint (13), we have for alli < k, j Æ £: 


gS) = (14) 


ð ð 
A= ia —— logg(¢) = aoe (Bijxelog(ai;ane) — Bijke log Bijke) 


= loga;; + log axe — log Bijke — 1 
and so 
Vi < k,j Al: Bijke = Caijake, where C = e77! | (15) 


Clearly, such Ø;jķe also satisfy the original constraints (12), and therefore the 
upper bound we obtain from this relaxation is in fact tight. 
To solve for C we sum (15) and use (13), getting 


55 Dijke = T QijAkl = 1-- = + 24% = = 


OA ix=k jHl 
Thus ČC = d/p and (15) becomes 


AA; Ke 


Vi < k, j Æl: Pijke = z 


(16) 
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Observe that p = p({a;;}) is the probability that a single edge whose end- 
points are chosen uniformly at random is properly colored by both ø and 7, if 
the overlap matrix is ai; = &ijn. Moreover, the values for the };;,¢ are exactly 
what we would obtain, in expectation, if we chose from among the (5) edges 
with replacement, rejecting those improperly colored by ø or 7, until we had 
dn/2 edges — in other words, if our graph model was G(n,m) with replacement, 
rather than Gra. 

Substituting the values (16) in (14) and applying (13) yields the following 
upper bound on g(¢): 


l PPTL a/a Evan see 0 
KO) < =e m 
( ) IL, a; f (d/p)2i<k.see Bijke Tic jpel@ijare) Paare 
Qij ; Q d/p 
1 4/2 I aij ikki xe ORE 
pA/? 
ij Xij i 
= Ja(n,m) ({aiz}) - 


In [4, Thm 5], Achlioptas and Naor showed that for d < c,_1 the function 
9G(n,m) iS maximized when a;; = 1/k? for all i, j. In this case p = (1 — 1/k)?, 
implying 


d 
Jmax S kep”? = k? (1 — z) 


and, therefore, that for some constant Cz and sufficiently large n 


dn 
E[X2] < Con7@-) k?” (1 — z) , 


7 Directions for Further Work 


A Sharp Threshold for Regular Graphs. It has long been conjectured that for 
every k > 2, there exists a critical constant c, such that a random graph 
G(n,m = en) is w.h.p. k-colorable ifc < c, and w.h.p. non-k-colorable if ¢ > cp. 
It is reasonable to conjecture that the same is true for random regular graphs, 
i.e. that for all k > 2, there exists a critical integer d, such that a random graph 
Gna is w.h.p. k-colorable if d < dp and w.h.p. non-k-colorable if d > dp. If this 
is true, our results imply that for d in “good” intervals (uk, ck) w.h.p. the chro- 
matic number of Gn a is precisely k + 1, while for d in “bad” intervals (Ck—1, Uk) 
the chromatic number is w.h.p. either k or k + 1. 


Improving the Second Moment Bound. Our proof establishes that if X, Y are 
the numbers of balanced k-colorings of Gn,a and G(n,m = dn/2), respectively, 
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then E[X]*/E[X?] = @(E[Y]?/E[Y?]). Therefore, any improvement on the up- 
per bound for E[Y?] given in [4] would immediately give an improved positive- 
probability k-colorability result for Gri. 

In particular, Moore has conjectured that the function gG(n,m) 1S maximized 
by matrices with a certain form. If true, this immediately gives an improved 
lowerbound, c%, for k-colorability satisfying cf_, > ux — 1. This would shrink 
the union of the “bad” intervals to a set of measure 0, with each such interval 
containing precisely one integer d for each k > ko. 


3-Colorability of Random Regular Graphs. It is easy to show that a random 
6-regular graph is w.h.p. non-3-colorable. On the other hand, in [1] the authors 
showed that 4-regular graphs are w.p.p. 3-colorable. Based on considerations 
from statistical physics, Krzakata, Pagnani and Weigt [8] have conjectured that 
a random 5-regular graph is w.h.p. 3-colorable. The authors (unpublished) have 
shown that applying the second moment method to the number of balanced 
3-colorings cannot establish this fact (even with positive probability). 
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Abstract. In standard property testing, the task is to distinguish be- 
tween objects that have a property P and those that are £-far from P, 
for some € > 0. In this setting, it is perfectly acceptable for the tester 
to provide a negative answer for every input object that does not satisfy 
P. This implies that property testing in and of itself cannot be expected 
to yield any information whatsoever about the distance from the object 
to the property. We address this problem in this paper, restricting our 
attention to monotonicity testing. A function f : {1,...,n} t+ R is at 
distance €¢ from being monotone if it can (and must) be modified at 
Efn places to become monotone. For any fixed ô > 0, we compute, with 
probability at least 2/3, an interval [(1/2 — d)e,¢] that encloses €f. The 
running time of our algorithm is O(e;' log log EF" log n), which is optimal 
within a factor of log log 7i and represents a substantial improvement 
over previous work. We give a second algorithm with an expected running 
time of O(e7' log n log log log n). 





1 Introduction 


Since the emergence of property testing in the nineties [12,8], great progress 
has been made on a long list of combinatorial, algebraic, and geometric testing 
problems; see [11,6,4] for surveys. Property testing is a relaxation of the standard 
decision problem: Given a property P, instead of determining exactly whether 
a given input object satisfies P or not, we require an exact answer only if the 
object satisfies the property or 1f it is far from doing so. This subsumes a notion 
of distance: Typically the object is said to be e-far from P if at least a fraction € 
of its description must be modified in order to enforce the property. The largest 
suche is called the distance of the object to P. In this setting, the tester can say 
“no” for every input object that does not satisfy P, which precludes the leaking 
of any information regarding the distance of the object to the property. 

This weakness has led Parnas, Ron, and Rubinfeld [10] to introduce the con- 
cept of tolerant property testing. Given 0 < £1 < €g < 1, a tolerant tester must 
accept all inputs that are not €,-farfrom P and reject all of those that are €g-far 
(and output anything it pleases otherwise). A related problem studied in [10] 
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is that of estimating the actual distance of the object to the property within 
prescribed error bounds. In the model considered, all algorithms are random- 
ized and err with probability at most 1/3. (or equivalently any arbitrarily small 
constant). 

Testing the monotonicity of functions has been extensively studied [1-3,5, 
7,9]. In the one-dimensional case, given a function f : {1,...,n} => R, after 
querying O(log n)/e function values, we can, with probability at least 2/3, ac- 
cept f if it is monotone and reject it if it is ¢-far from being monotone [3]. 
These methods do not provide for tolerant property testing, however. Very re- 
cently, Parnas, Ron and Rubinfeld [10] designed sublinear algorithms for tolerant 
property testing and distance approximation for two problems: function mono- 
tonicity and clustering. If €f denotes the distance of f to monotonicity, their 
algorithm computes an estimate éfor €p that satisfies (1/2)e, -6d <E<e7 +ô 
with high probability. The query complexity and running time of their algorithm 
are both O((logn)’/6*) (the O notation hides a factor of (loglogn)°“)). The 
algorithm maintains and queries a data structure called an “index-value tree.” 
Since the running time is sublinear, the tree is stored implicitly and only relevant 
portions are constructed whenever necessary, using random sampling to make 
approximate queries on the tree. Their construction is sophisticated and highly 
ingenious, but all in all quite involved. 

We propose a simpler, faster, algorithm that is nearly optimal. Given any 
fixeddé > 0, it outputs an interval [(1/2—4)e, £] that encloses €f with probability 
at least 2/3. The running time is O(eF' log log EF log n), which is optimal within 
a factor of log log ar (The optimality proof is quite simple and omitted from 
this version.) One thing to note is the different use of 6: in our algorithm it is 
part of the multiplicative factor, whereas in [10] it is an additive term. To achieve 
the same multiplicative factor as in our algorithm, the additive term needs to be 
@(de7). This makes the running time of Parnas et al.’s algorithm O((log n)" /e#), 
for any fixed 6. 

The starting point of our algorithm is the property tester of Ergun et al. [3], 
which relies on a key fact: There exist at least efn “critical” integers îi € 
{1,... n}; for i to be critical means that it is the (left or right) endpoint of 
an interval at least half of whose elements are in violation with 2. Here 2 is 
said to violate j if either i < j and f(z) > f(j) or i > j and f(t) < f(y). 
By proving an upper bound on the number of critical integers, we are able to 
define a “signature” distribution for f which reflects its distance €p fairly accu- 
rately. Specifically, two functions with distances to monotonicity off by a factor 
of 2 (roughly) will have signatures that are distinguishable in time O(e7' logn). 
This provides us with a tolerant property tester for monotonicity. We can turn 
it into a distance approximator by using a one-way searching strategy, which we 
discuss below. Just as in [10], our algorithm extends to higher dimension. 

We also present an improvement of our one-dimensional algorithm for small 
enough values of ¢. We show how to estimate €p in time O(e;* log n log log log n). 
Unlike in our previous algorithm, the number of steps in this one is itself a 
random variable; therefore, the running time is to be understood in the expected 
sense over the random bits used by the algorithm. 
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2 Estimating Distance to Monotonicity 


Given two functions f,g : {1,...,n} +» R, let d(f,g) = Prob[f(x) Æ g(x) 
denote the distance between f and g, where x € {1,...,} is chosen uniformly 
at random. We define £f = mingem d(f,g), where M is the set of monotone 
functions from {1,...,n} to R. 


Theorem 1. For any fixed 5 > 0, we can compute an interval {(1/2 — d)e,¢| 
that encloses eş with probability at least 2/3. The running time is 
O(e;? log log E7” log n}. 


It is not entirely clear from the theorem that amplifying the probability of 
success can be achieved by simply repeating the algorithm enough times and 
taking a majority vote. What if we get different candidate intervals every time? 
We do not. As will soon become obvious, majority voting does, indeed, boost 
the probability of success arbitrarily close to 1. 

It is easy to reduce the search for such an interval to a “distance separation” 
decision problem. Suppose that, given any € > 0, one can tell in O(e~! logn) 
time and with probability at least 2/3 whether €f > € or ef < (1/2 — d)e. If 
(1/2 — d)e < ef < g, the algorithm can report anything. For each k = 1,2,..., 
we run the algorithm clog(k + 1) times with € set to £x = (1/2 — 6)*, where 
cis a large enough constant, and we take a majority vote. We continue until 
we hear the report that ef > ce. By Chernoff ’s bound, the probability that 
E41 < Ef < €g; is at least 1 — J` >o O(1/ck?) > 2/3. The running time of 
Y icpce O(log(k+1))ez* logn, whichis O(e;" log log EF logn) time, as claimed. 

This does not quite do the job. Indeed, we are now left with the knowledge 
that eş falls in the interval [£¢+1,€e—1], which unfortunately is too big for our 
purposes. It is enclosed in the interval [£o/5, €o], for some 0 < €9 < 1, which we 
must now shrink to the right size. To do this we simply use the previous “distance 
separation” algorithm for the values (1 — kô)eo, for O < k < 1/6. This allows us 
to pinpoint ¢; within an interval of the form [(1/2 — O(6))e, €]. Rescaling ô gives 
us the desired result. It thus suffices to prove the following lemma: 


Lemma 1. For any fixed ¢,5 > 0, we can decide, in time O(e~' logn) and with 
probability at least 2/3, whether és > € ores < (1/2—d)e. If (1/2—d)e < ef <e, 
the algorithm can report anything. 


2.1 A Separation Oracle 


As mentioned in the introduction, the key to estimating the distance to mono- 
tonicity is to approximate the number of “critical” integers (to be defined in the 
next section). To identify a critical integer 7, we need to find an interval starting 
or ending at i such that there are many violations with 7 in the interval. This 
is done through random sampling, to ensure a sublinear running time. The mo- 
tivation for the following definitions on joint distributions of random variables 
will be made clear later in this section. 
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Let D be the joint distribution of m independent 0/1 random variables 
ZL1,-++;Xm, Which can be sampled independently. If Ex; < a for all ¿, then 
D is called a-light; else it is a-heavy. We describe an algorithm light-test which, 
given any a < b, determines whether a distribution is a-light or b-heavy. 


Lemma 2. If D is either a-light or b-heavy, for some fixeda < b, then with 
probability 2/3 we can tell which is the case in O(bm/(b — a)*) time. 


Proof. Call light-test({z1,...,2%m},Co), where co is chosen so that cı = colb — 
a)? /b is a large enough constant. The algorithm runs in time proportional to 
> E>0 Cok(m/2*) = O(com). To see why it works, we begin with a simple obser- 
vation. Suppose that Ex; > b, then at the k-th recursive call we sample zx; (if 
at all) exactly cok times; therefore, by Chernoff’s bounds, 


Prob[#; < (a + b)/2] = 279017) 


The same upper bound holds for the probability that ĉ; > (a + 6)/2, assuming 
that Ex; < a. Suppose now that : 


— D Is 0-HEAVY:Let x; be such that Ex; > b. At the k-th recursion call, the 
probability that S” is empty is 2~%*), Summing up over all k bounds the 
likelihood of erring by 1/3. 

— D IS a-LIGHT: The probability that any given 2; exceeds (a + b)/2 is at 
most 1/3 (conservatively) and so erring any time before the size of S is 
recursively reduced to below cı is doer <k<|S| Q—2k) — 9-8) < 1/6, Af- 
ter that stage, the probability of reaching ab-heavy verdict is at most 
O(c; (log c,)2~— @")) < 1/6. 

0O 


light-test (5, k) 


For each x € $, sample it k times and compute the average ; 
Form S = {x € S| > (a+ b)/2}. 


If |S'|=0, then output “a-light”. 
If |S’| >|S|/2, then output “b-heavy”. 
light-test(S’,k + co) 





2.2 Distance Separation: The Facts 


Given 0 < 6 < 1/2,the integer tis called ĝ-big if there exists 7 > i such that 


isk <j) < si) }] > (1/2-8)G-8+1) 
or, similarly, 7 <2 such that 


{9 SF Sis > FO }] > A/2- 5-5 +1). 
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Intuitively, integer 7 is big if f(z) violates monotonicity with an abundance of 
witnesses. In the following we show that when ô is small, the number of 6-big 
integers approximates €fn to within a factor of roughly 2. 


Lemma 3. (i) At least €sn integers are 0-big; (ii) no more than (2 + 46/(1 — 
26))esn integers are 6-big. 


Proof. Note that, for any 7 < j such that f(z) > f(j), either ¿i or 7 (or both) 
is O-big. Therefore, if we were to remove all the 0-big integers from the domain 
{1,... n}, the function f would become monotone; hence (i). 

To prove (11), let C be a set of epn integers in the domain of f over which 
the function can be modified to become monotone. An element 7 of C is called 
high-critical (resp. low-critical) if there is 7 ¢ C such that 7 > i and f(7) < f(t) 
(resp. j < i and f(j) > f(i)). Note that the two definitions are exclusive. For 
each 6-big i, we choose a unique witness 7; to its bigness (which one does not 
matter). If 7; > 2, then 2 is called right-big; else it is left-big. (Obviously, the 
classification depends on the choice of witnesses.) 

To bound the number of right-bigs, we charge low-criticals with a credit 
scheme. (Then we apply a similar procedure to charge left-bigs.) Initially, each 
element of C is assigned 1 credit. For each right-big 7 g C among n,...,1 in 
this order, spread one credit among all the low-criticals k such that i < k < ji 
and f(k) < f(t). We use the word “spread” because we do not simply drop one 
credit into one account. Rather, viewing the accounts as buckets and credits as 
water, we pour one unit of water one infinitesimal drop at a time, always pouring 
the next drop into the least filled bucket. (There are other ways to describe this 
charging scheme, none of them quite as poetic.) 

We now show that no low-critical ever receives an excess of 2 + 46/(1 — 26) 
credits. Suppose by contradiction that this were the case. Let 2 be the right- 
big that causes the low-critical k’s account to reach over 2 + 46/(1 — 26). By 
construction z is not low-critical; therefore, the excess occurs while right-big 2 is 
charging the l low-criticals k such that i < k < j; and f(k) < f(z). Note that, 
because i ¢ C, any k satisfying these two conditions is a low-critical and thus gets 
charged. With the uniform charging scheme (remember the water’), this ensures 
that all of these l low-criticals have the same amount of credits by the time they 
reach the excess value, which gives a total greater than 1(2 + 46/(1 — 26)). By 
definition of right-bigness, | > (1/2 — 6)(7; — i +1). But none of these accounts 
could be charged before step J:; therefore, 


(1/2 — ô) (Ji — i + 1)(2 + 40/(1 — 20)) < ji —i +1, 
which is a contradiction. 
We handle left-bigs in a similar way by running now from left to right, ie, 


i= 1,...,n. Since no integer can be both left-critical and right-critical, part (11) 
of the lemma follows. o 


2.3 Distance Separation: The Algorithm 
We need one piece of terminology before describing the distance separation al- 
gorithm. Given an interval in [u,v], we define two 0/1 random variables afu, v] 


234 Nir Ailon et al. 


and [u,v]: given random 7 € [u,v] N {1,...,n}, afu,v] = 1 (resp. Blu, v] = 1) 
iff f(u) > f(i)(resp. f(t) > f(v)). With probability at least 2/3, distance- 
separation( f, €, ô). reports that ef > (resp. er < (1/2 —4)e) if it is, indeed the 
case, and anything it wants if (1/2 —d)e < ey < €. 


distance-separation (f,<¢, 4) 


Pick s = [(1+6/2)e—'In2] random i€ {1,...,n}. 
For each 1<k<(5/d6)Inn, define oi) = ofi,i t+ (1+6/4)*] 


and xs) = pfi — (1+ 8/4)", a]. | | 

Let D be the distribution of (a, zs”, — a) i xy), sn al), as”, PEE, 
If D is (1/2—6/4)-heavy, then output “er>”. 

If D is (1/2—6/3)-light, then output “ep < (1/2 — je”. 





The algorithm assumes that both 6 and £€/ô are suitably small. The require- 
ment on ô is nonrestrictive. To make £ small, however, we use an artifice: set 
fli) = +œ for i = n + 1,...,O(n/ô). We also need to assume that the algo- 
rithm used for distinguishing between light and heavy succeeds with probability 
at least 1 — ô? (instead of 2/3); to do that iterate it log ô} times and take a 
majority vote. To prove the correctness of the algorithm, it suffices to show that: 


— Ife >e, then D is (1/2 — 6/4)-heavy with probability 1/2 + (4): 


By Lemma 3 (i), more than en integers are O-big, so the probability of 
hitting at least one of them in the first step (and hence, of ensuring that D 
is (1/2)/(1 + 6/4)-heavy) is at least 1 — (1 —¢)* > 1/2 + (6). 


— Ife, < (1/2 — d)e, then Dis (1/2 — 6/3)-light with probability 1/2 + (6): 


By Lemma 3 (ii), the number of 6/3-big integers is less than (1 — jen; 
therefore, the probability of missing all of them (and hence, of ensuring that 
Dis (1/2 — 6/3)-light) is at least (1 — (1 — d)e)* > 1/2 + (6). 


By running the whole algorithm O(1/67) times and taking a majority vote, 
we can boost the probability of success to 2/3. By Lemma 2, the running time 
is O(e~' logn), as claimed (for fixed 6). This completes the proof of Lemma 1 
and hence of Theorem 1. 


2.4 A Faster Algorithm for Small Distances 


We show in this section how to slightly improve the query complexity of the 
algorithm to 
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O(min {log loge; l log log log n} EF og n). 


The running time is now expected (over the random bits used by the algorithm). 
To do this, we need the following theorem: 


Theorem 2. We can compute an interval |Q(e/logn),¢] that encloses ep with 
probability at least 2/3. The expected running time is O(e7* log n). 


Using this theorem, it is clear that the factor log log EF" in the distance esti- 
mation algorithm can be replaced by min{log log TE log log logn}. Indeed, in- 
stead of taking k = 1,2,3,..., and running the separation oracle for each value of 
Ek a number of times (ie, clog(k+1) times), we redefine €, to be (1/2—6)*e, where 
€ is the estimate returned by Theorem 2. Because the maximum value of k is now 
O(log log n), the running time drops to O(min {log log EF log log log n}eze logn). 

To prove Theorem 2, we turn to a construction introduced by Goldreich et 
al. [7]. Define a subset P of pairs of integers: (i,j) € P if j > i, and j — i is at 
most t, where ¢ is the largest power of 2 that divides either 2 or 7. This set has 
the following two properties: 





— |P| = O(nlogn). 

— For any i < j, there exists k (i < k < j) such that both (i,k) € P and 
(k, 7) € P. This means, in particular, that for any violation (2,7) of f, there 
exists a “witness” (i,k) or (k,7) of the violation in the subset P. 


Now, for afunction f, let M be a maximum matching in the violation graph 
(the undirected graph whose vertex set is {1,...,n} and where ż¿ is connected 
to j ifi < j and f(z) > f(j)). It is known [7] that |M| = Ofe¢n); to be 
precise, lejn < |M| < epn. Let Q C P be the set of violations of f in P. 
Consider the bipartite graph G with M on the left and Q on the right. Connect 
an edge between (i,j) € M and (a,b) € Q if {i,j} N {a,b} # 0. By the second 
property above, and from the definition of a maximum matching, every node 
on the right has degree either 1 or 2, and every node on the left has degree at 
least 1; therefore, the cardinality of the right side is R(|M|). We would like to 
show that it is O(;/M|logn). If we could do that, then by sampling from P and 
checking for violations, we could then estimate the size of Q and get the desired 
approximation. Unfortunately, it is not quite the case that the cardinality of the 
right side is always O(|M| log n). To fix this problem, we need to introduce some 
more randomness. 

We slightly change the definition of P: for an integer r € [1,7] let P, denote 
the subset of pairs defined as follows: (i,j) € P, if j —2 is at most t, where t is 
the largest power of 2 that divides either i +r or ł +r. The set P, still has the 
two properties above. In addition, if r is chosen uniformly at random then, for 
any i, the expected number of 7 such that (i,j) € P, and j’ such that (j’,i) € P, 
is O(log n). The expected number of edges of the corresponding bipartite graph 
Gr, therefore, is O(|M|logn). So the expected cardinality of the right side is 
a|P,|,where œ € [R(ef/logn),O(ef)]. We sample P, to form an estimation 
a@for a and return £ = Caélogn, for some large enough constant C, to prove 
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Theorem 2. The estimation follows the predictable scheme: (1) pick a random 
r€{1,...,m}; (2) pick a pair (i, j) uniformly at random from P,; (3) if (i, 7) is 
a violation of f, output success, otherwise failure. The success probability is 
precisely a, so repeating the sampling enough times sharpens our estimation to 
the desired accuracy, as indicated by the following fact. 


Lemma 4. Given a O/I random variable with expectation a > 0, with probabil- 
ity at least 2/3, the value of 1fa can be approximated with a relative constant 
error by sampling it O(1/a) times on average. Therefore, œ canbe approximated 
within the same error and the same expected running time. 


Proof. Run Bernoulli trials on the random variable and define Y to be the num- 
ber of trials until (and including) the first 1. It 1s a geometric random variable 
with EY = 1/a, and var (Y) = (1—a@)/a? < (EY)?. By taking several samples 
of Y and averaging we get an estimate 1 / a of 1/a. Using Chebyshev’s inequality, 
a constant number of samples suffices to get a constant factor approximation. 
T 
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Abstract. Consider the following communication problem, that leads 
to a new notion of edge coloring. The communication network is repre- 
sented by a bipartite multigraph, where the nodes on one side are the 
transmitters and the nodes on the other side are the receivers. The edges 
correspond to messages, and every edge e is associated with an integer 
c(e), corresponding to the time it takes the message to reach its destina- 
tion. A proper k-edge-coloring with delays is afunction f from the edges 
to {0,1,...,4—1}, such that for every two edges e1 and e2 with the same 
transmitter, f(e1) Æ f(e2), and for every two edges e1 and e2 with the 
same receiver, f(e1)+c(e1) # f(e2)+c(e2) (mod k). Haxell, Wilfong and 
Winkler [10] conjectured that there always exists a proper edge coloring 
with delays using k = A + 1 colors, where A is the maximum degree of 
the graph. We prove that the conjecture asymptotically holds for simple 
bipartite graphs, using a probabilistic approach, and further show that 
it holds for some multigraphs, applying algebraic tools. The probabilistic 
proof provides an efficient algorithm for the corresponding algorithmic 
problem, whereas the algebraic method does not. 


1 Introduction 


Motivated by the study of optical networks, Haxell, Wilfong and Winkler con- 
sidered in [10] a communication network in which there are two groups of nodes: 
transmitters and receivers. Each transmitter has to send a set of messages, each 
of which should reach one receiver (more than one message per receiver is al- 
lowed). Each message has an associated delay, which is the time from the moment 
it is sent until it reaches its destination. The network is timed by a clock, so all 
times are integers. We wish to find a periodic setup of message sending for all 
transmitters, such that in each cycle all messages of all transmitters are sent, 
where each transmitter sends at most one message and each receiver gets at 
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most one message, at any given time unit. The objective is to find such a cycle 
of minimal length. 

We can formalize this problem as follows: we represent the network by a 
bipartite multigraph G = (V, E) with sides A and B, where the vertices of A are 
the transmitters and the vertices of B are the receivers. The edges correspond to 
the messages that are to be sent, and a function c : E — N associates each edge 
with its delay. The aim is to find the smallest number k for which there exists 
an edge coloring f : E — {0,1,...,k — 1}, such that for every two edges e and 
€2 that have a common vertex in A, f(e1) Æ f(e2), and for every two edges e1 
and ez that have a common vertex in B, f(e1) + c(e1) Æ f(e2) + c(e2) (mod k). 

The minimum number of colors is, clearly, at least A, where A is the maxi- 
mum degree in G. Furthermore, there are simple examples in which A+ 1 colors 
are required, for example, any A-regular graph in which all delays but one are 
0. Haxell, Wilfong and Winkler [10] raised the following conjecture. 


Conjecture I. Let G = (V, E) be a bipartite multigraph with sides A and B and 
with maximum degree A, and let c: E — N be a delay function. Then there is 
a coloring f : E — {0,1,..., A}, such that for every two edges e} and eg that 
have a common vertex in A, f(e,) Æ f(e2), and for every two edges eı and e2 
that have a common vertex in B, f(e,) + c(e1) £ f(e2) + c(e2) (mod (A+ 1)). 


In this paper, we show that the conjecture asymptotically holds for simple 
graphs. More precisely, we prove that if G is a simple bipartite graph with 
maximum degree A, then A + o0(A) colors suffice. Our proof uses a random 
coloring procedure, following the result of Kahn [12] and its extensions and 
variants by Häggkvist and Janssen [11] and Molloy and Reed [14,15] on list 
edge coloring. 

Using algebraic techniques, based on [2,3,7], we also show that the conjecture 
holds for some families of bipartite multigraphs such as all even length regular 
multi-cycles where the degree of regularity plus one is a prime. We further de- 
scribe a generalization of the problem to non-bipartite multigraphs, and prove 
the conjecture in some cases. 

In section 2 we outline the proof of the asymptotic result for simple bipar- 
tite graphs, in section 3 we present the algebraic proofs for some families of 
multigraphs. The final section 4 contains some concluding remarks. 


2 Simple Bipartite Graphs 


In this section we show that if G is a simple bipartite graph with maximum 
degree A, then there is a coloring with the required properties using (1+ 0(1))A 
colors. 


Theorem 1. For every e > 0 there isa Ap = Ao(e) such that the following 
holds for every A> Ao: 

Let G = (V, E) be a bipartite simple graph with sides A and B and with maximum 
degree A, and let e: E — N be a delay function. Then for k = [(1+.€)A] there 


Edge Coloring with Delays 239 


is a coloring f : E — {0,1,...,k —1}, such that for every two edges e, and ez 
that have a common vertex in A, f(e1) # f(e2), and for every two edges e, and 
e> that have a common vertex in B, f(e1) + c(e1) £ f (e2) + c(e2) (mod k). 


Throughout the section we omit all floor and ceiling signs, whenever these 
are not crucial. We use, throughout the paper, the notation O(x) to denote, as 
usual, O(a(log r)?). 

Let G = (V, E) be a simple bipartite graph with sides A and B, and assume 
without loss of generality that G is A-regular. Let c : E — N be the delay 
function, and let k = (1 + €)A, where e > 0 is an arbitrarily small constant. 
We show that if A is sufficiently large, then there is a coloring with the desired 
properties using k colors. 

We present a coloring procedure with three stages: 


1. Choose a small set of reserved colors for every edge. 

2. Iteratively color the edges as follows. In each iteration, assign every uncolored 
edge a random color from the unreserved colors that are still available for it. 
An edge retains the color assigned to it only if no adjacent edge is assigned 
a conflicting color. 

3. Complete the coloring from the lists of reserved colors. 


In the rest of this section we describe some of the details of each stage, and 
prove that, with positive probability, the procedure finds a coloring with the 
required properties. Due to space limitations, most of the technical details are 
postponed to the full version of the paper. 


2.1 Notation 


To simplify the description of the procedure, we extend the function c by defining 
for all e = (u,v) € E with u € A and v € B, c{e,u) = 0 and cfe,v) = ce). 
Now the coloring f should satisfy the condition that for every two edges e1, €2 € 
E with a common vertex u, f(e1) + c(e1,u) Æ f(e2) + e(e2,u), where all the 
operations on the colors are done modulo k. 

During the iterative procedure, we denote by Le C {0,1,...,k —1} the set 
of all unreserved colors still available for e, for all e € E. For all v € V let Ly = 
{c+c((u,v),v) | c € Liuw) (u,v) € E }. For a set of colors C and a color c, let 
C—c={c'-—c|c eC y}. Clearly, Lu.) E (Lu—e((u, v) u))N (Ly —e((u, v), v)), 
for all (u,v) € E. Thus, every time a color c is removed from a list Lẹ, then 
c — c(e,v) is removed from the lists Le, for all edges e incident with v. 

For all v € V and c € L,, denote by T,,. the set of all uncolored edges e 
incident with v, for which the color c — e(e, v) is still available, i.e. 


Toc = {e = (u,v) € E | c—c(e,v) € Le and e is uncolored }. 


In the first stage we choose a set of reserved colors for every edge. We do it by 
choosing a set of reserved colors Reserve, forall v € V, and then by defining for 
every edge e = (u,v) € E, Reserve, = (Reserve, —c(e, u)) (Reserve, —c(e, v)). 
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During the iterative procedure, for all v € V and color c, we denote by R,. 
the set of all uncolored edges e = (u,v) incident with v, such that the color 
c — c(e,v) + c(e, u) is reserved for u, Le. 


R, c = {e = (u,v) € E | c—c(e,v)+c(e,u) € Reserve, and (u,v) is uncolored}. 


For all i, e € E, v € V and color c, let 1%, li, tê „ and ri, , denote the sizes of 
the sets Le, Ly, Ty k: and Ry c, respectively, after i iterations. If after i iterations 
c ¢ Ly wedefine t „= 0. For all the above, i = 0 refers to the time before the 
first iteration, but “after the first stage of the procedure, in which the reserved 


colors are chosen. 


2.2 Coloring Scheme Overview 


We now describe the coloring procedure. Start by choosing a small set of reserved 
colors Reserve, for all v € V, and by defining for all e = (u,v) € E, Reserve, = 
(Reserve, — c(e,u)) N (Reserve, — c(e,v)). We would like these colors to be 
available for e at the final stage, and hence we remove from the initial set Le of 
every edge e = (u,v) € E, all the colors that conflict with the colors reserved 
for u and v. T beets for all e = (u,v) € E, the initial set Le is defined by 
L.={0;1,. — 1} \ ((Reserve, — c{e,u)) U (Reserve, = c(e,v))). We choose 


the reserved. ee so that for alle € E, I? > > (1+ 5)A, but we also make sure 


that for all e € E, |Reserve,| > <A, and for all v € V and color c, r?., < £A, 
so the coloring can be completed at the final stage. 

The second stage consists of a constant t = t(€) number of iterations. In each 
iteration, we assign to each edge e € E a color ec chosen randomly from Le. If 
there is a conflict, i.e. if there is an adjacent edge e’ to which we assigned a 
color c’ such that c+ c(e,v) = ce’ + c(e’,v), where v is the common vertex of e 
and e’, we uncolor e. If an edge e = (u,v) retains the color c it was assigned, 
then adjacent edges cannot be colored with conflicting colors. Thus we remove 
c+c(e,u)from Lu, c+ c(e,v) from Ly, and the corresponding colors from the 
lists Le of the edges e’ incident with u and v. 

We start with 12 > (1+ §)A, forall e € E, t$. < A and r}, < $4 forall 
v € V and color e. We show that the sizes of the’ sets Le and E decrease at 
roughly the same rate, and hence, for every i, the sizes ti , are slightly smaller 
than the sizes 1%. 

The t-th iteration is carried out as follows. We first remove colors from the 
sets Le so that they all have the same size, which we denote by l;—ı1. Now, for 
all e = (u,v) E€ E assign e a color c chosen randomly from Le. The color e is 
removed from e if an adjacent edge is assigned a conflicting color, 1.e. if there is 
an edge e incident with u that is assigned c + c(e,u) — c(e1,u), or an edge eg 
incident with v that is assigned c+ c(e,v) — c(e2,v). Therefore, the probability 
that e retains cis 


] 


i—1 


4z—1 i—l 
J umeteleu) t'v ctete.) ~2 ~ e72. 





(= 
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For every color c € Ly, the edges e incident with v that may retain c—c(e, v) 
are the edges in T,,,. For every e € T,,., the probability that e is assigned the 
color ¢ and retains it is roughly yx. There is at most one edge e € Ty, that 
retains c—c(e,v). Therefore, these events are disjoint, and if one of them occurs 
we remove cfrom L,. Hence, the probability that we remove c from L, is roughly 


i~1 
bye 


-2 
Fd e . 
l;_ je? 





For an edge e = (u,v) and a color c € Le, we remove cfrom Le if we remove 
c+c(e,u) from La or c+ c(e,v) from Ly. If these events were independent, the 
probability that we do not remove c from Le would have been roughly (1 ~e7?)?. 
We later show that although there is some dependence between those events, the 
probability that we do not remove c from Le is still close to (1 — e~*)?. Hence, 


Elle] E a 


For every v € V and c € Ly, if e = (u,v) € Tue, we remove e from Ty eif e 
retains the color it is assigned, or if e cannot be colored with c—c(e, v) anymore, 
since c—c(e,v)+c(e, u) was removedfrom Lau. If these events were independent, 
the probability that e remains in Ty e would have been roughly (1 — e~*)?, since 
the probability of each of these events is about e~*. We show later that the 
dependence between the above events does not change this probability much. 
Therefore, 

a dsten 

As for the sets Ry c, we remove an edge e from Re», only if it retains the 
color it was assigned. Therefore, 

E[r] eo (1 z E 

We execute the iterative procedure for t iterations, where we choose € so that 
with positive probability ome < A for all v € V and color c. We can then 
prove, using the local lemma (see, e.g., [4] chapter 5, or the results in [16,17,9]), 
that in this case the coloring can be completed with the reserved colors. 

Due to space limitations, the detailed proofs of the first and last stages are 
omitted, and will appear in the full vesions. In the next subsection we describe 
some of the details of the main part of our coloring scheme, the iterative proce- 
dure. 


2.3 The Iterative Procedure 


In each iteration, we assign every edge e a color c chosen randomly from Le. We 
let e retain c only if it causes no conflicts, i.e. no adjacent edge e’ is assigned 
c+ce(e,u)—c(e’, u), where u is the common vertex ofe and e’. If an edge e = (u, v) 
retains a color c, we remove c+c(e, u) from L, and c+c(e, v) from L,. To simplify 
the proof, we remove some colors from some of the lists Le at the beginning of 
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each iteration, so that all the lists have the same size, and we sometimes remove 
colors from edges and from the lists Ly even when there is no conflict, in order 
to equalize all the probabilities. The details are explained below. 

We show that the size of each Le and of each Te ,e decreases by a factor of 
approximately (1 — e~*)? in each iteration, and the size of each Ry e by a factor 
of approximately 1 — e~?. Each Ty e is initially slightly smaller than each Le, 
and since these sizes decrease at approximately the same rate, the above holds 
at the beginning of every iteration. 

Consider the i-th iteration, and assume /*-! > 1,_, for all e € E, and i. 
1; forall v € V and color c. We first remove colors from some of the lists Le so 
that they all have size exactly l;—ı. For each edge e = (u, v) and color c € Le, the 
probability that no adjacent edge is assigned a conflicting color, if e 1s assigned 
c, i.e. that no edge e; incident with u is assigned c + c(e, u) — c(e,,u), and that 
no edge eg incident with v is assigned c + c(e, v) — c(e2, v), is 


4-1 i—i] 
1 letele oere A 1 2l;1-2 9 
P(e,c) = [1- >{1- Fe, 


li—1 lii 








We now use an equalizing coin flip, so that if e is assigned c and no adjacent 
edge is assigned a conflicting color, we still remove c from e with probability 


1 


= e2P(e,c) an 


Eq(e,c) = 1 
This ensures that the probability that e retains c, conditional of e receiving c, 1s 


precisely 
P(e,c)(1 — Eq(e,c)) = e7?. 


For every v € V and c € Lẹ,, there is at most one edge e € Te, that retains 
c — c(e,v). Therefore, the events that e retains c — c(e,v) for all e € Tu e are 
pairwise disjoint, and hence the probability that we do not have to remove c 
from Ly, i.e. the probability that no edge e € Tu o retains c — c(e, v) is 


ii 
eS ae: 





=-j- 
Q(v, c) l,_1e2 Z 
If we do not have to remove c from L,, we still remove it with probability 
bee 
Vq(v,c) = 1- ——~. 
=I Ta 


This ensures that the probability that c remains in Lẹ is precisely 


Q(v,c)(1 — Valu, e)) = 1 — e°. 


Lemma 1. For every v € V andcolor c, Elri = (1 —e7?)ri-! 


Proof. An edge is removed from R», if and only if it retains the color it is 
assigned. This happens with probability e~*. Therefore, Efri e} = (1—e7*)ri7J. 


o 
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Lemma 2. For eache € E, E[Ù] > (1 — e? 
Lemma 3. For every v € V andcolorc € Ly, Elti a] < (1 — e7?)?ti + e7’. 


The proofs of Lemmas 2 and 3 require a more careful analysis, and will appear 
in the full version. 

In Lemmas 1, 2 and 3, we have shown that the expected size of every Le 
and Ty, decreases in one iteration by a factor of approximately (1 — e~*)*, and 
of every Rv c by a factor of approximately 1 — e~*. We wish to prove that with 
positive probability, the sizes of all these sets indeed decrease by approximately 
these factors. For this purpose we show that these values are highly concentrated. 
More precisely, we prove that there exists a constant 8 > 0, such that the 
following hold for all 1 <2 <t. 


Lemma 4. For eache€ E, 


ee l| - E(l.]| > log Avii- 1 sie Pee, 


where the probability is over the random choices of the i-th iteration. 





Lemma 5. For eachv EV andc€ Ly, if 2 > l-1 then 


Pr [|ti e- Ele cll > aa < e-2iog? A, 
where the probability is over the random choices ofthe i-th iteration. 


Lemma 6. Foreachv €V andcolorc, ifry > Ea then 
; ; A 2 
Pr ee — Efr: | > log A,/r?,.| S erie eS 


where the probability is over the random choices of the i-th iteration. 


The proofs of Lemmas 4, 5 and 6 use Talagrand’s inequality (see, e.g. [4,15]), 
and will appear in the full version. 

To complete the proof we apply the local lemma to each iteration, to show 
that with positive probability, the size of every set is within a small error of the 
expected size. 


Lemma 7. With positive probability, the following hold after i iterations, for all 
O54 <1; 


1. For allee E P . 
$ > (1 — e7?) (14 5) A-O6(VA). 


2. For allv € V and color c 


tie S(1—e?)"A + O(VA). 
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3. For allv € V and color c 
rie <(1-e7’)' =A +Õ(VA). 


Proof. We prove the lemma by induction on 7. Suppose the claim holds after 
i — 1 iterations. Note that by the induction hypothesis, aoe < 1;_,. Thus, we 
can apply all the lemmas that were proved in this section. For all v € V and 
color c,if ti’ < (1 — e7?” A or ri} < (1 — e~*)'€A then the corresponding 
condition after 7 iterations trivially holds. Otherwise, we can apply Lemmas 5 
and 6. 

For all e € E, let Ae be the event that l? violates the first condition, and 
for all v € V and color c, let Byc and Cy, be the events that ie o and rj, 
violate the second and third conditions, respectively. By Lemmas 4, 5 and 6, the 
probability of each one of these events is less than e Blog’ A for some constant 
B. Every event A, is mutually independent of all other events except for those 
that correspond to the vertices incident with e and their neighbours, and to 
edges incident with these vertices. Thus, it is mutually independent of all but 
O(A?) of the other events. Every event By,- or Cu e is mutually independent of 
all other events except for those that correspond to the edges incident with v, to 
the vertices incident with them and to the neighbours of these vertices. Thus, it 
is mutually independent of all but O(A*) of the other events. Thus, by the local 
lemma, all the conditions hold with positive probability. 

To complete the proof of the lemma we use the bounds on 19, t? , and r? 


e? “v,e v,e 


given by the first stage of our procedure. O 


3 Algebraic Methods 


In this section we prove Conjecture 1 for several families of graphs using algebraic 
methods. The relevance of results from additive number theory to this conjecture 
appears already in [10], where the authors apply the main result of Hall [8] to 
prove their conjecture for the multigraph with two vertices and d parallel edges 
between them. Here we consider several more complicated cases. 


3.1 Even Multi-cycles 


Consider the case where the graph G is a d-regular multigraph whose underlying 
simple graph is a simple cycle of even length. We show that if d+ 1 is a prime 
then there exists a coloring with the required properties using d+ 1 colors. 

In our proof, we use the following theorem proved in [3]: 


Theorem 2. (Combinatorial Nullstellensatz) 

Let F be an arbitrary field, and let P = P(x1,...,2%n) be a polynomial in 
F[x1,..-,2n]. Suppose that the degree of P is X`;_,ti, where each t; is a non- 
negative integer, and suppose the coefficient of | [;_; xt in P is nonzero. Then, 
if S1,..., Sn are subsets of F with |S;| > t;, there ares, € $1, 82 E S2, ..., 8n E Sn 
so that P(s,...,8n) #9. 
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Let G = (V, E) be a simple even cycle with, possibly, multiple edges. Let A 
and B be the sides of G, and denote the vertices of A by aj,a9,...,a,, and the 
vertices of B by b1,b2,...,bn, such that there are edges between a; and b; for 
all 1 < i < n, between b; and a;4; for all 1 < ¿è < n — 1, and between 6, and 
a,. Since Gis d regular, all the edges (a;,b;) have the same multiplicity s, and 
hence all the other edges have multiplicity t = d — s. 

We now associate a polynomial P with the graph G. For every edge e, we 
have a variable ze. For every two edges e; and eg which have a common vertex 
in A we have a term (ze, — Ze,), and for every two edges e} and e3 which have 
a common vertex in B we have a term (£e, + c(e1) — Ze, — c(e2)). Thus, the 
polynomial P is defined by 


P= [[ (er - ep) [| (er + e(e1) — ze, — c(e2)) 


eiNeoN AZM e1NeaN BZ 


Since the graph is d-regular, every edge has d — 1 terms for each one of its 
vertices. Hence, the total degree of P is nd(d — 1). 


Proposition 1. Jf d+ 1 is a prime then the coefficient of the monomial 
Ikee z?—! in P is nonzero modulo d +1. 


Proof. The monomial Į [scg z¢—1 is of maximum degree, and thus its coefficient 


in P is equal to its coefficient in the polynomial 


Q = I] (Te, = Tez) I] (Le, i Tez) : 


eqne2znNA¥d eyNegNBHAG 


In [7] (see also [2]) it is shown that, for any d-regular planar multigraph, the ab- 
solute value of this coefficient is equal to the number of proper d-edge-colorings. 
Every d-edge-coloring of G is obtained by partitioning the colors into a subset 
of size s and a subset of size t, and for every two connected vertices, choosing 
a permutation of the appropriate set. Thus the number of edge colorings with 
d colors is (4)(s!)"(¢!)” = dl(s!)"-1(¢!)"-1, which is nonzero modulo d + 1 since 
d+ 1 is a prime. X 


Corollary 1. Let G = (V, E) be an even length d-regular multi-cycle, where 
d+ 1 is a prime. Let (A,B) be a bipartition of G, and letec: E — N be a delay 
function. Then, there is a coloring f : E — {0,1,...,d} such that ife and e3 
have a common vertex in A then f(e1) # f (e2), and ifeı and ez have a common 
vertex in B then f(e1) + c(e1) # f (e2) + c(e2) (mod (d + 1)). 


Proof. By Theorem 2, with Se = {0,1,...,d} and te = d — 1 for every e E€ E, 
there is a function f : E — {0,1,...,d} such that the value of P, when every 
Ze is assigned the value f(e), is nonzero modulo d + 1. Thus, for every two 
edges e} and ez that have a common vertex in A, f(e1) #4 f (e2) since we have a 
term (£e; — Ze) in P, and for every two edges eı and ez that have a common 
vertex in B, f(e1) + e(e1) # f (e2) + c(e2) (mod (d + 1)) since we have a term 
(£e, +c(e1) — Le, — c(€2)) in P. O 
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3.2 Multi-K4 


The problem can be generalized to non-bipartite multigraphs. In this case, we 
specify for every edge which endpoint is the transmitter and which is the receiver. 
Following the notations used in section 2, for every edge e = (u,v), where u is 
the transmitter and v is the receiver, we define c(e,u) = 0, and c(e,v) to be the 
delay associated with e. 

We can further generalize the problem as follows. Given a multigraph G = 
(V,E), where every edge e = (u,v) € E is associated with two integers c(e, u) 
and c(e,v), we aim to find the smallest number k such that there is a coloring 
f: E— {0,1,...,4—1}, satisfying the following. For every vertex v and every 
two edges e; Æ ez incident with v, f(e1) + c(e1,v) Æ f(e2) + c(e2,v) (mod k). 
In this case, A+ 1 colors do not always suffice, simply because x’/(G) may be as 
large as [3A], where x/(G) is the edge chromatic number of G. The following 
conjecture seems plausible for this case. 


Conjecture 2. Let G = (V, E) be a multigraph, and suppose that every edge 
e = (u,v) € E is associated with two integers c(e,u) and c(e,v). Let k = 
x'(G) + 1. Then there exists a coloring f : E — {0,1,...,4 —1} such that for 
every vertex v and every two edges e; Æ e2 incident with v, f(e,) + c(e1,v) Æ 


f(e2) + c(e2,v) (mod k). 


Proposition 2. Let G = (V, E) be a d-reqular multi-K4, where every edge e = 
(u,v) E€ E is associated with two integers c(e,u) and c(e,v), and suppose d+1 is 
a prime. Then there is acoloring f : E — {0,1,...,d}, satisfying the condition 
that for every vertexu and every two edges e€, # eg incident withv, f(e,) + 
c(e1,v) Æ f(e2) + c(e2,v) (mod (d+1)). 


The proof is similar to the proof of the result for even cycles. 


Proof. First, note that if G is d-regular, then every pair of non-adjacent edges 
of K4 have the same multiplicity in G. Denote these multiplicities by a, b and c. 
We associate the following polynomial with G: 


P= I] I] (Le, + e(€1, v) — Le, — c(e2,v)). 


vEV e1,4#e2EE : vEeiNez 


P is a polynomial in 2d variables, with total degree 2d(d—1). Hence, by Theorem 
2, if the coefficient of [ [se g x?—1 is nonzero modulo d+1, then there is a function 
f: E — {0,1,...,d}, such that if every variable ze is assigned f(e) then the value 
of P is nonzero modulo d+ 1, and therefore f satisfies the required properties. 


The coefficient of ] Leg z¢—1 in P is equal to its coefficient in 


Q = I] I] (Le, = Wez) 


vEV €1 Fey CE: veerf leg 


which is, by [7], the number of proper d-edge-colorings of G. Such a coloring 
is obtained as follows. First, choose a permutation of the colors and color the 
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edges incident to a vertex u accordingly. Now for every pair of other vertices v 
and w, the set of colors that may be used to color the edges between v and w 
is the same set used for the edges between u and the forth vertex, and we only 
choose the permutation of this set. Thus, the number of proper d-edge-colorings 
is dlalb!c! £ 0 (mod (d+ 1)), since d + 1 is a prime. 0 


4 Concluding Remarks 


In section 3 we proved Conjecture 1 for some graphs using algebraic techniques. 
This method can be used to prove the conjecture for several other graphs. How- 
ever, it seems that in order to prove the conjecture for the general case, more 
ideas are needed. In the graphs for which we used Theorem 2, the theorem im- 
plies that there is a proper edge coloring with delays using A + 1 colors, even if 
there is one forbidden color for every edge. 

In section 2 we showed that Conjecture 1 asymptotically holds for simple 
bipartite graphs. It would be interesting to extend this proof to multigraphs as 
well. 

In the probabilistic proof presented in section 2, we proved that A + o(A) 
colors suffice, and did not make any effort to minimize the o(A) term. By mod- 
ifying our proof slightly, we can show that A + O(A?/3) colors suffice, and it 
seems plausible that a more careful analysis, following the discussion in [14], can 
even imply that A + Õ(V â) colors suffice. 

The probabilistic proof can be extended to other variations of edge coloring. 
Instead of the delays, one can associate with every edge e and an endpoint v of 
e an injective function ge, on the colors. Then, by our proof, there is an edge 
coloring f using A + 0(A) colors such that for every vertex v and any two edges 
e1 # e2 incident with v, Je1,v(f(e1)) F gea w (f (€2)). 

The known results about the algorithmic version of the local lemma, initiated 
by Beck ([5], see also [1],[13], [6]), can be combined with our probabilistic proof in 
Section 2 to design a polynomial time algorithm that solves the corresponding 
algorithmic problem. In contrast, the algebraic proofs of Section 3 supply no 
efficient procedures for the corresponding problems, and it will be interesting to 
find such algorithms. 
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Abstract. A quantum encryption scheme (also called private quantum channel, 
or state randomization protocol) is a one-time pad for quantum messages. If two 
parties share a classical random string, one of them can transmit a quantum state 
to the other so that an eavesdropper gets little or no information about the state 
being transmitted. Perfect encryption schemes leak no information at all about 
the message. Approximate encryption schemes leak a non-zero (though small) 
amount of information but require a shorter shared random key. Approximate 
schemes with short keys have been shown to have a number of applications in 
quantum cryptography and information theory [8]. 

This paper provides the first deterministic, polynomial-time constructions of 
quantum approximate encryption schemes with short keys. Previous construc- 
tions [8] are probabilistic — that is, they show that if the operators used for en- 
cryption are chosen at random, then with high probability the resulting protocol 
will be a secure encryption scheme. Moreover, the resulting protocol descriptions 
are exponentially long. Our protocols use keys of the same length as the proba- 
bilistic constructions; to encrypt n qubits approximately, one needs n + o(n) bits 
of shared key [8], whereas 2n bits of key are necessary for perfect encryption [3]. 
An additional contribution of this paper is a connection between classical combi- 
natorial derandomization and constructions of pseudo-random matrix families in 
a continuous space. 


1 Introduction 


A quantum encryption scheme (or private quantum channel, or state randomization 
protocol) allows Alice, holding a classical key’, to scramble a quantum state and send 
it to Bob (via a quantum channel) so that (1) Bob, given the key, can recover Alice’s state 
exactly and (2) an adversary Eve who intercepts the ciphertext learns nothing about the 
message, as long as she doesn’t know the key. We do not assume any shared quantum 
states between Alice and Bob, nor any back channels from Bob to Alice’. 


" A.A. supported by NSF grant DMS-0111298. A.S. supported by a Microsoft Ph.D. Fellowship. 

' Classical keys are inherently easier to store, distribute and manipulate, since they can be 
copied. More subtly, encryption with a shared quantum key is in many ways a dual problem to 
encryption with a classical key; see [8,5] for more discussion. 

2 A back channel from Bob to Alice would allow using quantum key distribution to generate a 
long secret key. However, such interaction is often impossible, e.g. if Alice wants to encrypt 
stored data for her own later use. 


K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 249-260, 2004. 
© Springer-Verlag Berlin Heidelberg 2004 
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There are two variants of this definition. An encryption scheme is called perfect 
if Eve learns zero information from the ciphertext, and approximate if Eve can learn 
some non-zero amount of information. A perfect encryption ensures that the distribu- 
tions (density matrices) of ciphertexts corresponding to different messages are exactly 
identical, while an approximate scheme only requires that they be very close; we give 
formal definitions further below. In the classical case, both perfect and approximate en- 
cryption require keys of roughly the same length — n bits of key for n bits of message. 
In the quantum case, the situation is different. 

For perfect encryption, Ambainis et al. [3] showed that 2n bits of key are necessary 
and sufficient to encrypt n qubits. The construction consists of applying two classical 
one-time pads — one in the “standard” basis {|0),|1)} and another in the “diagonal” 
basis {+ (|0) + |1)), 45(10) — |1))}. 

Approximate encryption was studied by Hayden, Leung, Shor and Winters [8]. They 
introduced an additional, useful relaxation: they showed that if the plaintext is not en- 
tangled with Eve’s system to begin with, then one can get approximate quantum en- 
cryption using only n + o(n) bits of key — roughly half as many as are necessary for 
perfect encryption’. The assumption that Eve’s system is unentangled with the message 
is necessary for this result; otherwiseroughly 2n bits are needed, even for approximate 
encryption. The assumption holds in the quantum counterpart of the one-time pad situ- 
ation (one party prepares a quantum message and sends it to the second party, using the 
encryption scheme) as long as the message is not part of a larger cryptographic protocol. 

Hayden et al. [8] showed that a random set of 2"+°'™ unitary matrices leads to a 
good encryption scheme with high probability (to encrypt, Alice uses the key to choose 
one of the matrices from the set and applies the corresponding operator to her input). 
However, verifying that a particular set of matrices yields a good encryption scheme 
is not efficient; even writing down the list of matrices is prohibitive, since there are 
exponentially many of them. 

This paper presents the first polynomial time constructions of approximate quantum 
encryption schemes (to relish the oxymoron: derandomized randomization protocols). 
The constructions run in time O(n?) when the message p consists of n qubits. That is, 
given the key and the input message, Alice can produce the output using O(n?) steps 
on a quantum computer. The key length we achieve is slightly better than that of the 
probabilistic construction of [8]. Our results apply to the trace norm on matrices; exact 
results are stated further below. 

The main tools in our construction are small-bias sets [10] of strings in {0, 1}?”. 
Such sets have proved useful in derandomizing algorithms [10], constructing short 
PCPs [6] and the encryption of high-entropy messages [12]. Thus, one of the contri- 
butions of this paper is a connection between classical combinatorial derandomization 
and constructions of pseudo-random matrix families in a continuous space. Specifically, 
we connecti Fourier analysis over C22" to Fourier analysis over the matrices C?” <2", 
This parallels to some extent the connection between quantum error-correcting codes 
over n qubits and classical codes over GF'(4)”. 


> The result of [8] highlights an error in the proof of a lower bound on key length of authentica- 
tion schemes in [4]. The results of that paper remain essentially correct, but the definition of 
authentication requires some strengthening, and the proof of the lower bound is more involved. 
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Definitions. We assume that the reader is familiar with the basic notation of quantum 
computing (see [11] for an introduction). Syntactically, an approximate quantum en- 
cryption scheme is a set of 2* invertible operators { E,.|K € {0,1}*}. The E,,’s may be 
unitary, but need not be: it is sufficient that one be able to recover the input p from the 
output E,.(e), which may live in a larger-dimensional space than p. Each Ex takes n 
qubits as input and produces n’ > n qubits of puar If n’ = n then each operator Ex 
corresponds to a unitary matrix Ux» that is Ex (p) = U,.pU. 

For an input density matrix” p, the density matrix of the ciphertext from the adver- 
sary’s point of view is: 


E(p) =Ex[Ex(ol=a¢ Y Erlo) 
ine 1}* 
When the scheme is length-preserving, this yields €(p) = a Y Ux pUt. 


Definition 1. The set ofoperators {Ex} is an approximate quantum encryption scheme 
with leakage € (also called “e€-randomizing scheme”) for n qubits if 


for all density matrices p on n qubits: D(E(p), = 5p7 ll) = |é() - = r| <e. 
tr 


Here I refers to the identity matrix in dimension Qn" and D(.,-) refers to the trace 
distance between density matrices. The trace norm of a matrix ø is the trace of the 
absolute value of o (equivalently, the sum of the absolute values of the eigenvalues). 
The trace distance between two matrices p,o is the trace norm of their difference: 





A 
D(p,0) = |le = olli = Trl — ol) 


This distance plays the same role for quantum states that statistical difference plays 
for probability distributions: the maximum probability of distinguishing between two 
quantum states p, g via a single measurement is 3 + +D(p, g). One can also measure 
leakage with respect to other norms; see below. 


Remark 1. This definition of quantum encryption implicitly assumes that the message 
state p is not entangled with the adversary’s system. Without that assumption the def- 
inition above is not sufficient, and it is not possible to get secure quantum encryption 
using n(1 + o(1)) bits of key (roughly 2nbits are provably necessary”). Thus, this sort 
of construction is not universally applicable, and must be used with care. 





Previous Work. Ambainis et al. [3] considered perfect encryption; this corresponds to 
the case where e = 0. The choice of matrix norm is irrelevant there, since €(p) = sar. 
As mentioned above, they showed that 2n bits of key are necessary and sufficient. The 


* Recall that for a pure state |p), the density matrix p is |¢) (J. 

` This folklore result appears more or less explicitly in both [4,8]. Similar arguments show that 
n bits of key are necessary to encrypt n classical bits, even with access to quantum computers 
(but not interaction). 
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construction uses the key to choose one of 2?” Pauli operators (defined below) and 
applies that to the input state. 

Hayden et al. [8] showed that a set of O(n2"/e*) unitary operators suffices. They 
showed this both for the trace norm, and for the “operator norm,” discussed below. For 
the trace norm, they also showed that a random set of Pauli matrices of the same size 
would suffice. This means that for encrypting n qubits, they gave a non-polynomial- 
time, randomized scheme requiring n + logn + 2 log(1/e) + O(1) bits of key. 


Our Results. We present three explicit, polynomial time constructions of approximate 
state randomization protocols for the trace norm. All are based on existing constructions 
of 6-biased sets [10,2,1], or on families of sets with small average bias. The three 
constructions are explained and proven secure in Sections 3.1, 3.2 and 3.3, respectively. 

The first construction is length-preserving, and requires n + 2 logn + 2 log(1/e) + 
O(1) of key, roughly matching the performance of the non-explicit construction. 
The second construction is length-increasing: it encodes n qubits into n qubits and 2n 
classical bits but uses a shorter key: only n + 2 log(1/e) bits of key are required. Both 
of these constructions are quite simple, and are proven secure using the same Fourier- 
analytic technique. 

The final construction has a more sophisticated proof, but allows for a length- 
preserving scheme with slightly better dependence on the number of qubits: 


n + min {2logn + 2log(1/e), logn + 3 log(1/e)} + O(1) 


bits of key. The right-hand term provides a better bound when € > L, 


Randomization Schemes for Other Norms? Definition 1 measures leakage with re- 
spect to the trace norm on density matrices, ||-||,,. This is good enough for encryption 
since the trace norm captures distinguishability of states. However, Hayden et al. [8] 
also considered randomization schemes which give guarantees with respect to a differ- 
ent norm, the operator norm. 

A guarantee on the operator norm implies a guarantee for the trace norm, but 
schemes with the operator norm guarantee also have a host of less cryptographic appli- 
cations, for example: constructing efficient quantum data hiding schemes in the LOCC 
(local operation and classical communication) model; exhibiting “locked” classical cor- 
relations in quantum states [8]; relaxed authentication of quantum states using few bits 
of key [9]; and transmitting quantum states over a classical channel using n + o(n) bits 
of communication, rather than the usual 2n bits required for quantum teleportation [5]. 

More formally, for a d x d Hermitian matrix A with eigenvalues {Aj, Ag, ..., Aa}, the 
operator norm (or oo-norm) is the largesteigenvalue, || Al], = max |A,|, the Frobenius 
norm is the Euclidean length of the vector of eigenvalues, ||Al|, = (>>, \?7)/2, and the 
trace norm is the sum of the absolute values of the eignvalues, |All, = 5°; |A|. It is 
easy to see the chain of inequalities: 


lAl < VaIAllo < dlll- 
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We can then state the condition for a map € to be €-randomizing map for n qubits 
in three forms of increasing strength. For all input states p on n qubits: 


EO- All, <6 NEW- ll, VA EW - Hla < €/2" 


Our constructions satisfy the definition with respect to the Frobenius norm, but they 
are not known to satisfy the stronger operator-norm definition. This suggests two inter- 
esting questions. First, is it possible to prove that the other applications of state random- 
ization schemes require only a guarantee on the Frobenius norm? Second, is it possible 
to design explicit (i.e. polynomial-time, deterministic) randomization schemes that give 
good guarantees with respect to the operator norm? 


The remainder of this paper describes our constructions and their proofs of security. 


2 Preliminaries 


Small-Bias Spaces. The bias of a random variable A in {0,1}” with respect to a string 
a € {0,1}” is the distance from uniform of the bit a© A, where © refers to the standard 
dot product on Z7: 


A(a) = Eg [(—1)°°4] = 2Prla © A = 0] —1. 


The function A is the Fourier transform of the probability mass function of the distri- 
bution, taken over the group Z5. 

The bias of a set S € {0,1}” with respect to œ is simply the bias of the uniform 
distribution over that set. A set S is called 6-biased if the absolute value of its bias is at 
most 6 for all a 4 0”. 

Small-bias sets of size polynomial in n and 1/6 were first constructed by Naor and 
Naor [10]. Alon, Bruck et al. (ABNNR, [1]) gave explicit (i.e. deterministic, polynomial- 
time) constructions of 5-biased sets in {0,1}” with size O(n/d*). Constructions with 
size O(n? / ô?) were provided by Alon, Goldreich, et al. (AGHP, [2]). The AGHP con- 
struction is better when ô = o(1/n). In both cases, the i*” string in a set can be con- 
structed in roughly n? time (regardless of 6). 

One can sample a random point from a 6-biased space over {0,1}” using either 
logn + 3log(1/8) + O(1) bits of randomness (using ABNNR) or using 2logn + 
2 log(1/6) bits (using AGHP). 


Small-Bias Set Families. One can generalize small bias to families of sets (or random 
variables) by requiring that on average, the bias of a random set from the family with 
respect to every a is low [7]. Specifically, the expectation of the squared bias must be at 
most 62. Many results on 6-biased sets also hold for 6-biased families, which are easier 
to construct. 


Definition 2. A family of random variables (or sets) {Ai},e, is 6-biased if 


Eir |Ai(a)?| < 6? forall a #0". 
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Note that this is not equivalent, in general, to requiring that the expected bias be less 
than 6. There are two important special cases: 


1. If S is a 6-biased set, then {S} is a 6-biased set family with a single member; 

2. A family of linear spaces {C;},.; is 6-biased if no particular word is contained in 
the dual C+ of a random space C; from the family with high probability. Specifi- 
cally: 

A 0 if a g CH 
Gaee an cL 


Hence a family of codes is 5-biased if and only if Pris{a € CH] < 6%, for every 
a Æ 0”. Note that to meet the definition, for linear codes the expected bias must be 
at most 62, while for a single set the bias need only be ô. 


One can get a good ĝ-biased family simply by taking {C;} to be the set of all linear 
spaces of dimension k. The probability that any fixed non-zero vector @ lies in the dual 
of a random space is exactly ô? = a =a , which is at most 27*. 

One can save some randomness in the choice of the space using a standard pairwise 
independence construction. View {0,1}" as GF'(2”), and let K C GF(2") be an 
additive subgroup of size 2*. For every non-zero string a, let the space C, be given 
by all multiples ak, where x € K, and the product is taken in GF'(2”). The family 
{Ca | a € GF(2"), a Æ 0} has the same bias as the set of all linear spaces (5 < 2—*/?), 
To see this, let {K1,...,4~} be a basis of K (over GF(2)). A string a is in C+ if and 
only ifa © (akı) = --- = a © (ak,) = 0. This is a system of k linearly independent 
constraints on a, and so it is satisfied with probability 52 = 27% when a — GF(2"), 
and even lower probability when we restrict a to be non-zero. Choosing a set Ca from 
the family requires n bits of randomness. 





Entropy of Quantum States. As with classical distributions, there are several ways to 
measure the entropy of a quantum density matrix. We’ll use the analogue of collision 
entropy (a.k.a. Renyi entropy). 

For a classical random variable A on {0, 1}”, the collision probability of two inde- 
pendent samples of X is pe = >>, Pr[A = a]*. The Renyi entropy of A is — log pe. 

For a quantum density matrix p, the analogous quantity is — log Tr(p?). If the eigen- 
values of p are {pz}, then the eigenvalues of p? are {p?}, and so Tr(p7) is exactly the 
collision probability of the distribution obtained by measuring p in a basis of eigenvec- 
tors. \/ Tr(p2) is called the Frobenius norm of p. 

Ifp is the completely mixed state ind dimensions, p = 51, then Tr(p*) is1/d. The 
following fact states that any other density matrix for which this quantity is small must 
be very close to I. The fact follows by applying the (standard) inequality Tr(|A])? < 
dTr( A”) to the Hermitian matrix A = p — I/d. 


Fact 1. [fp is d-dimensional quantum state and Tr(p”) < 4(1+€?), then D(p, 41) < €. 


Pauli Matrices. The 2 x 2 Pauli matrices are generated by the matrices: 


r-(83) 7=(8) 
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The Pauli matrices are the four matrices {1, X, Z, X Z}. These form a basis for 
the space of all 2 x 2 complex matrices. Since XZ = -ZX, and Z? = X? = 1, 
the set generated by X and Z is given by the Pauli matrices and their opposites: 
(EL eX EZ XZN: 

If u and v are n-bit strings, we denote the corresponding tensor product of Pauli 
matrices by X“Z”. That is, if we write u = (ul, ..., un) and v = (v1, ..., Un ), then 


AL SX OAL. 


(The strings x and z indicate in which positions of the tensor product X and Z 
appear, respectively.) The set {X,,Z, | u,v € {0,1}"} forms a basis for the 2” x 2” 
complex matrices. The main facts we will need are given below: 


1. Products of Pauli matrices obey the group structure of {0,1}*” up to a minus sign. 
That is, (X"%Z")(X°Z°) = (—1)2O°X¥ee Zv, 

2. Any pair of Pauli matrices either commutes or anti-commutes. Specifically, 
(X"Z?)(X°Z*) a (—1)#Ob+vOe (XIZ (Xe Z. 

3. The trace of X” Z” is 0 if (u, v) 4 0?” (and otherwise it is Tr(I) = 2”). 

4. (XZ) = ZX" =- "ze 


Pauli Matrices and Fourier Analysis. The Pauli matrices form a basis for the set of all 
2” x 2” matrices. Given a density matrix p, we can write p = } u ve{o,1}r Qun X“ Z”. 
This basis is orthonormal with respect to the inner product given by 5x Tr(AÎB ), where 
A, B are square matrices. That is, se Tr((X“ Ze) Xe Z’) ET on 

Thus, the usual arithmetic of orthogonal bases (and Fourier analysis) applies. One 
can immediately deduce certain properties of the coefficients œu,» in the decomposi- 
tion of a matrix p. First, we have the formula a,» = sw Tr(Z”X “o). Second, the 
squared norm of p is given by the squared norm of the coefficients, that is sx Ir(p! p)= 
Juv |Qu,v|*. Since p is a density matrix, it is Hermitian (pt = p). One can use this 
fact, and the formula for the coefficients a,,,, to get a compact formula for the Renyi 
entropy (or Frobenius norm) in terms of the decomposition in the Pauli basis: 


1 u v 
Tr(o”) = zz 2 TX" Zp)’. 


3 State Randomization and Approximate Encryption 


3.1 Encrypting with a Small-Bias Space 


The ideal quantum one-time pad applies a random Pauli matrix to the input [3]. Con- 
sider instead a scheme which first chooses a 2n-bit string from some set with small bias 
5 (we will set 6 later to be €27”/2). If the set of strings is B we have: 


E(po) = La XO X°Z’poZ’X" = Eas [X° Z" p02? X"| 


A (a,b)EB 
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That is, we choose the key from the set B, which consists of 2n-bit strings. To encrypt, 
we view a 2n-bit string as the concatenation (a, 5) of two strings of n bits, and apply 
the corresponding Pauli matrix. 

(The intuition comes from the proof that Cayley graphs based on €-biased spaces 
are good expanders: applying a Pauli operator chosen from a 6-biased family of strings 
to Po will cause all the Fourier coefficients of po to be reduced by a factor of 6, which 
implies that the “collision probability” (Frobenius norm) of po also gets multiplied by 
6. We expand on this intuition below.) 

As a first step, we can try to see if a measurement given by a Pauli matrix X” Z” 
can distinguish the resulting ciphertext from a totally mixed state. More explicitly, we 
perform a measurement which projects the ciphertext onto one of the two eigenspaces 
of the matrix X“Z”. We output the corresponding eigenvalue. (All Pauli matrices have 
two eigenvalues with eigenspaces of equal dimension. The eigenvalues are always either 
—| and 1 or—2z and 2.) 

To see how well a particular Pauli matrix X“Z” will do at distinguishing, it is 
sufficient to compute 


ITr(X"Z"E(p0)). 


This is exactly the statistical difference between the Pauli measurement’s outcome 
and a uniform random choice from the two eigenvalues. We can compute it explicitly: 


Tr(X*Z°E(p0)) = Tr (X"Z"Ea bjen [X°Z"poZ’X*}) 
= Ea b [Tr( X“ Z” X" Z? pp Z°X?)| 
= Ea,» [Tr(Z° X° X” Z” X° Z’ po)] 
aas (AD OTO] Tr X" 2" fo) 


Since a © uv + b © wis linear in the concatenated 2n-bit vector (a, b), we can take 
advantage of the small bias of the set B to get a bound: 


ITr(X"“Z°E(po))| < ô|Tr( X" Z” po)! when (u,v) 4 0°” 


Equivalently: if we express pọ in the basis of matrices X“ Z”, then each coefficient 
shrinks by a factor of at least 6 after encryption. We can now bound the distance from 
the identity by computing Tr(E(p0)?): 


Tr(E(p0)®) = = SIT X"Z"E(00)) ? 


1 6? eZ) 2 1 29n 2 
Sota SO {Tr(X“Z"po)|? < 5a (1 + 6°2"Tr(9)) 


S gn 
(u,v) #02" 


Setting 6 = €2~"/?, we get approximate encryption for all states (since Tr(p2) < 1). 
Using the constructions of AGHP [2] for small-bias spaces, we get a polynomial-time 
scheme that uses n + 2log n + 2 log(1/e) bits of key. 
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3.2 A Scheme with Shorter Key Length 





We can improve the key length of the previous scheme using 6-biased families of sets. 
The tradeoff is that the resulting states are longer: the ciphertext consists ofn qubits and 
2n classical bits. In classical terms, the encryption algorithm uses additional random- 
ness which is not part of the shared key; in the quantum computing model, however, 
that randomness is “free” if one is allowed to discard ancilla qubits. 


Lemma 1. If {A;};ez is a family of subsets of {0, 1}?” with average square bias 6°, 
then the operator 


E(po) = Eiez [li)(i] 8 Ease a, [X°2Z" p02°X"]] 
is an approximate encryption scheme for n qubits with leakage € whenever 6 < €27"/?, 


Before proving the lemma, we give an example using the small-bias set family from 
the preliminaries. View the key set {0,1}* as an additive subgroup K of the field F = 
GF(2?"). For every element a € F, define the set Ca = {ax|x € K}. The family 
{Ca} has bias 6 < 2—*/? (Section 2). The corresponding encryption scheme takes a 
key x € {0,1}* C GF(2?”): 


Choose a —pr GF(2?") \ {0} 

Compute the product ax € GF(2?") 

Write ax as a concatenation (a, b), where a,b € {0,1}” 

Output the classical string a and the quantum state X°Z°p9Z°X? 


E(po; K) = 


With a quantum computer, random bits are not really necessary for choosing œ; it is 
sufficient to prepare 2n EPR pairs and discard one qubit from each pair. For the scheme 
to be secure, the bias 6 should be less than ,/e/2", and so the key only needs to be 
n + 2log(1/e) bits long. The main disadvantage is that the length of the ciphertext has 
increased by 2n classical bits. 


Proof. As before, the proof will use elementary Fourier analysis over the hypercube 
Z2”, and intuition comes from the proof that Cayley graphs based on €-biased set fam- 
ilies are also expanders. 

Think of the output of the encryption scheme as a single quantum state consisting 
of two systems: the first system is a classical string describing which member of the 
6-biased family will be used. The second system is the encrypted quantum state. To 
complete the proof, it is enough to bound the collision entropy of the entire system by 
FT (1 + €?). 

For each ¿ € Z (that is, for each member of the set family), let p; denote the encryp- 
tion of pọ with a random operator from the set A;. The first step of the proof is to show 
that the collision entropy of the entire system is equal to the average collision entropy 
of the states p;. 


Claim. Tr(E(po)*) = Eis [Tr(p?)] 
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Proof. We can write E(po) = a >_; li G| Q pi. Then we have 
Tr(E(po)*) = aed A Dij Tr((|2) (219) (9) Q Pipi) 
Since (illj) = ô: j, we get Tr(E(po)*) = Ze X; Tr(p?), as desired. o 
Take any string w = (u,v) € {0,1}2", where u,v € {0,1}”. Recall that Â; (u, v) 


is the ordinary Fourier coefficient (over Ze) of the uniform distribution on A;, that is 
A;(u,v) = Eaa; [((—1)*©”]. From the previous proof, we know that 


Tr(X%Z" pi) = A;(v,u) - Tr( X” Z” po). 


We can now compute the average collision entropy of the states p;. Using linearity 
of expectations: 


E: [Tr(o?)] =E | A+ A D XZ p) 


(u,v) £0 
=et+h DO E [|Tr(X"2Z"p,)7] 
(u,v) #0 
=k te DO E [Ai(v,w)?| [TX Zp)? 
‘Ge v) #0 


The expression E; Ait, u)?| is exactly the quantity bounded by the (squared) bias 6?. 
As in the previous proof, the entropy Tr(E(po)*) is bounded by FIT] (1 + 672" Tr(p2)). 





By our choice of ô, the entropy 1 ITI (1 + €), and so E(p2) is within trace 
distance € of the completely mixed state. O 


3.3 Hybrid Construction 


Let d be a prime between 2” and 2”+1, Then, it suffices to show how to randomize a 
state in a d-dimensional space Ha spanned by |i}, i € {0,1,...,d — 1}, since a state 
on n qubits can be embedded into Hg. We define X and Z on this space by X|j) = 
I(j + 1) mod d) and Z|j) = e?7*9/4|5\, Notice that X7Z* = e?**0*)/4 Zk XI and 
(XIZ*)t = Z-* XJ, (The definitions of X and Z are different than in the previous 
sections, since we are operating on a space of prime dimension). 

We start with a construction that uses n + 1 bits of randomness and achieves ap- 
proximate encryption for € = 1. (Notice that this is a non-trivial security guarantee. The 
trace distance between perfectly distinguishable states is 2. Distance 1 means that the 
state cannot be distinguished from i with success probability more than 3/4.) We will 
then extend it to any € > 0, using more randomness. 


Let 
14 1 


E(p) = io Xe" ieee, ae 
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Claim. 


Tr(E(p)?) < 4(1 + Tr(o%)). 


Proof. Let p = E(p). 


Tir = 2 Pyl (o) = 2 Pul (at S > palea 


i jij 


The first sum is equal to dł = 4 because pj, = ae pkk = 4. To calculate the 
second sum, we split it into sums St = $}; Pi ipt(Pi ipe) fort = 1,2,...,d — 1. (In 
the indices for p;; and Pij» we use 2 + t as a shortcut for (¢ + t) mod d.) We have 


d—1 
/ at 
Pi itt = d S40" Pi-a,i-a+t) 
a==0 
where w is the d®® root of unity. 
1 {42 
/ } x 2 b?—a7)t * 
Pi ire (Pitted eee?) ` |Pi+a,i+t+a|" + > whe) Pi—a,i+t-a(Pi—b,i+t—b) 
a=0 a,b ab 
Therefore, 
-15 Pi, al To d2 D Ci jPi i+t (Pij) 
tF5 
where 


cij = 5 wtite)?—G+a)*)t _ 5 wit -I +2ali—i))t a yl- > pre-e 


Since d is a prime, 2(i — j)t is not divisible by d. Therefore, X, w2*20-J* = 0, 
d 


ES 


ty D lil’ 


iF) 


Tr((p’)? 


By Fact 1, D(E(p), 2) <1. 

We now improve this construction to any €. Let B be an €-biased set on m = [log d] 
bits. For b € {0,1}, define a unitary transformation Uy as follows. Identify numbers 
0,1,...,d—1 withstrings x € {0,1}™. Define Ulz) = (—1)°©* |z), with bO z being 
the usual (bitwise) inner product of b and x. (Note that Up is just to the Z operator over 
a different group. It is the same Z operator used in the previous sections). Let 


E'(p) = ) | UrpUy and E"(p) = E(E"(p)). 
bE B 
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We claim that E£” is €-approximate encry puon pean W.1.o.g., gata that p is a pure 
state |b) = J; cili). Then pi; = cic}. Let p' = J bE U,pU; be the result of 
encrypting p by £’. Then, 


1 

—_ = bOr+bOy 5 ia 1)PO(e+4) 5 

Pov = Tg Zg” = LOY 
be B be B 


Since B is e-biased, |p y| < €|Pxy| for any x,y, £ # y. Therefore, >’, 7, Pryl S 

€ J rzy |Pzyl. Together with the Claim above and Fact 1, this implies that E” is €- 
randomizing. m number of key bits used by E” is n + log |B| + O(1) which is n + 
2logn + 2log t = + O(1) if the AGHP scheme is used and n + logn + 3 log + + O(1) 
if ABNNR is used. The first bound is the same as the one achieved by using small-bias 
spaces directly (Section 3.1). The second bound gives a better result when € > L, 


Acknowledgements 


We are grateful for helpful discussions with and comments from Claude Crépeau, 
Daniel Gottesman, Patrick Hayden, Debbie Leung, Sofya Raskhodnikova, Alex Samo- 
rodnitsky, and an anonymous referee. 


References 


1. Noga Alon, Jehoshua Bruck, Joseph Naor, Moni Naor, and Ronny Roth. Construction of 
asymptotically good low-rate error-correcting codes through pseudo-random graphs. IEEE 
Transactions on Information Theory, 38:509-516, 1992. 

2. Noga Alon, Oded Goldreich, Johan Hastad, René Peralta. Simple Construction of Almost 
k-wise Independent Random Variables. Random Structures and Algorithms 3(3): 289-304. 

3. Andris Ambainis, Michele Mosca, Alain Tapp, Ronald de Wolf. Private Quantum Channels. 
FOCS 2000: 547-553. 

4. Howard Barnum, Claude Crépeau, Daniel Gottesman, Adam Smith, Alain Tapp. Authenti- 
cation of Quantum Messages. FOCS 2002: 449-458. 

5. Charles Bennett, Patrick Hayden, Debbie Leung, Peter Shor and Andreas Winter. Remote 
preparation of quantum states. ArXiv e-Print quant-ph/0307100. 

6. Eli Ben-Sasson, Madhu Sudan, Salil P. Vadhan, Avi Wigderson. Randomness-efficient low 
degree tests and short PCPs via epsilon-biased sets. STOC 2003: 612-621. 

7. Yevgeniy Dodis and Adam Smith. Encryption of High-Entropy Sources. Manuscript, 2003. 

8. Patrick Hayden, Debbie Leung, Peter Shor and Andreas Winter. Randomizing quantum 
states: Constructions and applications. Comm. Math. Phys., to appear. Also ArXiv e-print 
quant-ph/0307104. 

9. Debbie Leung, personal communication, 2004. 

10. Joseph Naor, Moni Naor. Small-Bias Probability Spaces: Efficient Constructions and Appli- 
cations. SIAM J. Comput. 22(4): 838-856 (1993). 

11. Michael Nielsen, [Isaac Chuang. Quantum Computation and Quantum Information. Cam- 
bridge University Press, 2000. 

12. Alexander Russell, Hong Wang. How to Fool an Unbounded Adversary with a Short Key. 
EUROCRYPT 2002: 133-148. 


The Sketching Complexity of Pattern Matching 


Ziv Bar-Yossef, T.S. Jayram, Robert Krauthgamer, and Ravi Kumar 


IBM Almaden Research Center, 650 Harry Road, San Jose, CA 95120, USA 


{ziv, jayram, robi, ravi }@almaden.ibm.com 


Abstract. We address the problems of pattern matching and approxi- 
mate pattern matching in the sketching model. We show that it is im- 
possible to compress the text into a small sketch and use only the sketch 
to decide whether a given pattern occurs in the text. We also prove a 
sketch size lower bound for approximate pattern matching, and show it 
is tight up to a logarithmic factor. 


1 Introduction 


Pattern matching is the problem of locating a given (smaller) pattern in a (larger) 
text. It is one of the most fundamental problems studied in computer science, 
having a wide range of uses in text processing, information retrieval, computa- 
tional biology, compilers, and web search. These application areas typically deal 
with large amounts of data and therefore necessitate highly efficient algorithms 
in terms of time and space. 

In order to save space, I/O, and bandwidth, large text files are frequently 
stored in compressed form. The naive method for locating patterns in compressed 
files is to first decompress the files, and then run one of the standard pattern 
matching algorithms on them. Amir and Benson [2] initiated the study of pat- 
tern matching in compressed files; their approach is to process the compressed 
text directly, without first decompressing it. Their algorithm, as well as all the 
subsequent work in this area [3, 21, 12, 24, 11, 22, 15], deal with lossless compres- 
sion schemes, such as Huffman coding and the Lempel-Ziv algorithm. The main 
focus of these results is the speedup gained by processing the compressed text 
directly. 

In this paper we investigate a closely related question: how succinctly can 
one compress a text file into a small “sketch”, and yet allow locating patterns 
in the text using the sketch alone? In this context we consider not only lossless 
compression schemes but also lossy ones. In turn, we permit pattern matching 
algorithms that are randomized and can make errors with some small constant 
probability. Our main focus is not on the speed of the pattern matching al- 
gorithms but rather on the succinctness of the compression. Highly succinct 
compression schemes of this sort could be very appealing in domains where the 
text is a massive data set or when the text needs to be sent over a network. 

A fundamental and well known model that addresses problems of this kind 
is the sketching model [8,14], which is a powerful paradigm in the context of 
computations over massive data sets. Given a function, the idea is to produce 
a fingerprint (sketch) of the data that is succinct yet rich enough to let one 
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compute or approximate the function on the data. The parameters that play a 
key role in the applications are the size of the sketch, the time needed to produce 
the sketch and the time required to compute the function given the sketch. 


Results. Our first main result is an impossibility theorem showing that in the 
worst-case, no sketching algorithm can compress the text by more than a con- 
stant factor and yet allow exact pattern matching. Specifically, any sketching 
algorithm that compresses any text of length n into a sketch of size s and en- 
ables determining from the sketch alone whether an input pattern of length 
m = §2(logn) matches the text or not with a constant probability of error re- 
quires s > R(n — m). We further show that the bound is tight, up to constant 
factors. 

The proof of this lower bound turns out to be more intricate than one might 
expect. One of the peculiarities of the problem is that it exhibits completely 
different behaviors for m < (1 — o(1)) logn and m > logn. In the former case, 
a simple compression of the text into a sketch of size 2” is possible. We prove 
a matching lower bound for this range of m as well. These results are described 
in Section 3. 

Our second main result is a lower bound on the size of sketches for approxi- 
mate pattern matching, which is a relaxed version of pattern matching: (1) if the 
pattern occurs in the text, the output should be “a match”; (11) if every substring 
of the text is at Hamming distance at least k from the pattern, the output should 
be “no match”. An arbitrary answer is allowed if neither of the two holds. We 
prove that any sketching algorithm for approximate pattern matching, requires 
sketch size (2(n/m), where n is the length of the text, m is the length of the pat- 
tern, and the Hamming distance at question is k = em, for a fixed0 < £ <1. We 
further show that this bound is tight, up to a logarithmic factor. These results 
are described in Section 4. 

Interestingly, Batu et al. [6] showed a sampling procedure that solves (a 
restricted version of) approximate pattern matching using O(n/ m) non-adaptive 
samples from the text. In particular, their algorithm yields a sketching algorithm 
with sketch size O(n/m). This procedure was the main building block in their 
sub-linear time algorithm for weakly approximating the edit distance. The fact 
that our sketching lower bound nearly matches their sampling upper bound 
suggests that it might be hard to improve their edit distance algorithm, even in 
the sketching model. 


Techniques. A sketching algorithm naturally corresponds to the communication 
complexity of a one-way protocol. Alice holds the text and Bob holds the pattern. 
Alice needs to send a single message to Bob (the “sketch’’), and Bob needs to 
use this message as well as his input to determine whether there is a match or 
not’. 

' Usually, a sketching algorithm corresponds to the communication complexity of a 
simultaneous messages protocol, which is equivalent to summarizing each of the text 
and the pattern into a small sketch. However, in the context of pattern matching, it 
is reasonable to have a weaker requirement, namely, that only the text needs to be 
summarized. 
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The most classical problem which is hard for one-way communication com- 
plexity is the indexing function: Alice is given a string x € {0,1}” and Bob is 
given an index i € {1,...,n}, and based on a single message from Alice, Bob has 
to output x,;. It is well known that in any protocol solving this problem, even 
a randomized one, Alice’s message has to be of length R(n). Our lower bound 
for approximate pattern matching is proved by a reduction from the indexing 
function. 

Our lower bound for exact pattern matching uses a reduction from a variant 
of the indexing function. In this variant, Alice gets a string x € {0,1}"; Bob gets 
an index i € [n] and also the m — 1 bits preceding x; in x; the goal is to output 
Xi. Using tools from information theory we prove an R(n — m) lower bound for 
this problem in the one-way communication complexity model. 


Related Work. Pattern matching and approximate pattern matching have a rich 
history and extensive literature — see, for instance, the excellent resource page 
[20]. To the best of our knowledge, pattern matching, has not been considered in 
the sketching model. For approximate pattern matching, the only relevant result 
appears to be the above mentioned work of Batu et al. [6]. 

Sketching algorithms for various problems, such as estimation of similarity 
between documents [8,7,9], approximation of Hamming distance [19,13] and 
edit distance [4] between strings, and computation of Lp distances between vec- 
tors [1,14], have been proposed in the last few years. Sketching is also a useful 
tool for approximate nearest-neighbor schemes [19,16], and it is related to low- 
dimensional embeddings and to locality-sensitive hash functions [16]. 


2 Preliminaries 


2.1 Communication Complexity 


A sketching algorithm is best viewed as a public-coin one-way communication 
complexity protocol. Two players, Alice and Bob, would like to jointly compute 
a two-argument function f : X x Y — Z. Alice is given xz € Æ and Bob is 
given y € Y. Based on her input and based on randomness that is shared with 
Bob, Alice prepares a “sketch” s,4(x) and sends it to Bob. Bob uses the sketch, 
his own input y, and the shared randomness to determine the value f(x,y). 
For every input (z,y) E€ X x Y, the protocol is required to be correct with 
probability at least 1 — 6, where 0 < 6 < 1 is some small constant. Typically, 
the error probability 6 can be reduced by repeating the procedure several times 
independently (in parallel). 

The main measure of cost of a sketching algorithm is the length of the sketch 
s(x) on the worst-case choice of shared randomness and of the input x. Another 
important resource is the amount of randomness between Alice and Bob. New- 
man [23] shows that the amount of shared randomness can always be reduced 
to O(log #) at the cost of increasing the protocol’s error probability by ô’. In 
one-way protocols, Alice can privately generate these O(log |) and send them 
to Bob along with the sketch s4(z). 
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Some of our lower bounds use a reduction from the standard indexing prob- 
lem, which we denote by IND;: Alice is given a string x € {0,1}', Bob is 
given j € ft], and the goal is to output x;. This problem has a lower bound 
of t(1 — H2(6)) in the one-way communication complexity model [18, 5]. 


2.2 Pattern Matching and Approximate Pattern Matching 


For a Boolean string x € {0,1}” and integer 1 < j < n, let z; denote the jth 
bit in x. For integers 1 < i < j < n, [i,j] denotes the corresponding integer 
interval, [n] the interval [1,n] = {1,...,}, and x[2, j] denotes the substring of x 
that starts at position 7 and ends at position 7. We define the pattern matching 
and approximate pattern matching problems in the communication model. 

Let 0 < m < n. In the (n,m) pattern matching problem, denoted PMn,m, 
Alice gets a string x € {0,1}” and Bob gets a string y € {0,1}. The goal is to 
determine whether there exists an index i € [»—m-+1] such that x[i, i+m-—1] = y. 
For the purpose of lower bounds, we would consider the simple Boolean function 
defined above. However, some of the algorithms we present can additionally find 


the position of the match 2, if it exists. 


We denote the Hamming distance of two strings x,y € {0, 1}” by HD(x, y) 2 


{i € [n] : x; # y,}|. A relaxed version of pattern matching is the (n,m, e€) 
approximate pattern matching problem, denoted APMn,m,s, in which Bob would 
like to determine whether there exists an index i € [n — m + 1] such that x[i, i + 
m—1] = y, or whether for all i € [n], HD(x[t,i-+m-—1], y) > em, assuming that 
one of the two holds. 


Notation. Throughout the paper we denote random variables in upper case. For 
a Boolean string x € {0,1}”, |x| denotes the Hamming weight (i.e., the number of 
1’s) of x. log denotes a logarithm to the base 2; In denotes the natural logarithm. 
H2(p) = —plogp — (1 — p) log(1 — p) is the binary entropy function. 


3 Exact Pattern Matching 


In this section we obtain a simple sketching algorithm for exact pattern matching 
and show almost matching lower bounds. Recall that we denote by PM, m the 
problem in which Alice gets a string x € {0,1}”, Bob gets a string y € {0,1}”, 
and their goal is to find whether there exists an index i € [n — m+ 1] such that 
xli i +m- 1] =y. 


3.1 Upper Bounds 


First, we show an efficient (randomized) sketching algorithm for the pattern 
matching problem, based on the Karp—Rabin hash function [17]. Next, we show 
a deterministic sketching algorithm for the Boolean version of the pattern match- 
ing problem. 
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Proposition 1. Form < n—logn, there is a one-sided error randomized sketch- 


ing algorithm for the pattern matching problem PMn, m using a sketch of size 
O(n- m). 


Proof. The randomized algorithm is based on the Karp—Rabin method [17]. Let 
t = n—m+ 1; we assume in the sequel that t < n/3, as otherwise the proof follows 
trivially by Alice sending x. Let xt,...,xt denote the sequence of t substrings 
of x of length m. Alice and Bob use the shared randomness to pick a (weak) 2- 
universal hash function h : {0,1} — [n?]. Alice sends to Bob A(x'),..., A(x*). 
Bob outputs “match found at 3”, if h(x‘) = h(y). If no such i exists, Bob outputs 
“no match found”. 

This is a one-sided error algorithm: if there is a match, it will surely be output. 
There is a possibility for a false match, though: when x’ Æ y, but h(x’) = h(y). 
The probability for a false match is thus at most the probability k has a collision 
between y and any of {x!,...,x°}. A union bound shows that since the range of 
his large enough, the probability of a collision between y and any zê is at most 
O(1/n). 

The scheme described above uses a sketch of size O(t log n) = O((n—m) logn). 
A further improvement is possible using the Karp—Rabin hash function: h(b) = 
(So, bi- 2™-*) mod p, where p is a randomly chosen prime in the range [n°]. 
The advantage of this hash function is that the value of h(x*t!) can be com- 
puted from the value of h(x‘) and from the two bits x; and Xim: h(x*t!) = 
((h(x*t) — x; -2™—1)-2+x;,m) mod p. Thus, what Alice needs to send is only 
h(x"), the first t bits of x, and the last t bits of x. Thus, the sketch size goes 
down to 2t + O(logn) = O(n — m). 


Proposition 2. There is a deterministic sketching algorithm for the pattern 
matching problem PMn.m using a Sketch of size 2”. 


Proof. In the deterministic algorithm Alice sends to Bob a characteristic vector 
of length 2” specifying all the strings of length m that occur as substrings of x. 
Bob outputs “match found” if and only if y is one of the substrings indicated by 
the characteristic vector. 


3.2 Lower Bounds 


We show lower bounds on the sketch size for the pattern matching problem. 
The first one, Theorem 1, deals with the case m > ((logn). The second one, 
Theorem 2, considers the case m < O(log n). 


Theorem 1. Jf n <m+62™, then any 6-error randomized sketching algorithm 
for the pattern matching problem PMn m requires a sketch of size atleast (n — 
m+ 1): (1 ~ H2(26)), where Ho(-) is the binary entropy function. 


Proof. Using Yao’s Lemma [25], it suffices to exhibit a distribution pz over in- 
stances of PMn,m, and prove that any deterministic sketching algorithm that 
computes PM,,, correctly with probability at least 1 — 6 when running over 
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inputs chosen according to u requires a sketch of size at least (n — m + 1) - (1 — 
Hp(26)). 

The distribution jz is defined as follows. Alice is given a uniformly chosen 
bitstring X € {0,1}”. Bob is given a uniformly chosen substring of X of length 
m — 1 concatenated with the bit 1. 

The distributional lower bound w.r.t. 1s proven via a reduction from the 
following version of the indexing function, which we denote by IND,,;: Alice is 
given a string x € {0,1}”, and Bob is given an index k+ 1 < j < n and a string 
y € {0,1}*, which is promised to be equal to the substring x[j — k, j — 1]. The 
goal is to compute xj. 

Let v be the following distribution over instances of IND,,,. Alice gets a 
uniformly chosen bitstring X in {0,1}"; Bob gets an index J, which is cho- 
sen independently and uniformly in the interval [k + 1,n], and also the bits 
pe eee, A Aen 

The following lemma shows the reduction. 


Lemma 1. Any deterministic sketching algorithm IT that computes PMn m with 
error probability at most 6 on instances drawn according to pu yields a determin- 
istic sketching algorithm IT’ that computes INDn,m—1 with error probability at 
most 26 on instances drawn according to v and using exactly the same sketch 
size. 


Proof. In the indexing algorithm M’, given an input x € {0,1}”, Alice sends 
whatever message she would have sent on this input in the algorithm J. Given 
his input (j, y), where m < j < n and y € {0,1}™~1, Bob simulates the role of 
Bob in the pattern matching algorithm J on the input yo 1, where o denotes 
the concatenation of strings. If the output in J is “match found”, Bob outputs 
“1” and otherwise he outputs “0”. 

It is easy to verify that when the input given to M’ is distributed according 
to v, then the input given to M in the reduction is distributed according to p. 
We can thus assume that J errs with probability at most ô. 

Fix an input (x, (j,y)) for IND; m—1, for which the protocol H is correct on 
(x,yo1).If INDpm_i(x, (j, y)) = 1, then the string yo 1 is a substring of x (at 
position j — m + 1), and thus /7’ will output “1”, as needed. Suppose then that 
IND» m_1(x, (J, y)) = 0. Clearly, yo 1 does not equal to the substring of x that 
starts at position 7 — m+ 1. Therefore, IM’ outputs “0”, unless there is some 
other substring of x that happens to equal to yol. We next prove that the latter 
occurs with low probability. 

Define E to be the set of instances (x, (j,y)), for which there exists some 
i Æ j, so that the substring xfi, i +m — 1] equals y o 1. The proof of Lemma 1 
would follow easily once we prove that Pr(E) < ô, since M’ errs on an input 
(x, (i, y)) only if either it belongs to E or if JT errs on it. 


Proposition 3. Pr(E) < ô. 


Proof. Pr(E) can be rewritten as Pr(3i A J : Xfi, i+m-—1] = X|[J-m+1, J- 
1] o 1). In order to bound Pr(E), it would suffice to show that for all choices of 
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m<j<n, Pritt #7 : Xji,i +m -— 1] = X|j—m+1,j—1 01]) <6. So for 
the rest of the argument, fix such a 7. 

Define t © j — m + 1. We will show that for all i # t, Pr(X[i, i + m — 
1] = X[t,7 — 1| o 1) < 1/2™. It would then follow from the union bound that 
Pr(3i #7 : Xfi,it+m—1] =X[j—m4t1,j—1 olf) < (n—m)/2" <6. 

Fix any i # t. Suppose, for example, 7 < t (the case 7 > ¢ is dealt with 
similarly). X[i,2-+m—1] = Xft, j — 1]o1 if and only if X[i,i+m—2] = X[t, 7-1] 
and Xi4m-—1 = 1. It is easy to verify that the two events are independent and that 
the probability of the second is 1/2. Thus, it suffices to show Pr(X[2,i+m— 2] = 
X[t, j 1) = 1/2". 

We will denote X* = X[i,i + m ~ 2] and Xt = X[t,j — 1]. Let p be the 
length of the longest prefix of X’ that does not overlap X* (p can be anywhere 
between 1 and m — 1). Divide X’ and Xt into s = [(m — 1)/p| non-overlapping 
blocks of size p each (except, maybe, for the last one which is shorter). Call 
the corresponding substrings X;,...,X‘% and X‘,...,X%, respectively. Note that 
Xi = X{_, for q =2,...,s. X* = X’ if and only if X = Xj, for all q =1,...,s, 
or equivalently, if and only if all the s blocks of Xt equal to X$ (except, maybe, 
the last one which needs to be a prefix of X$). The latter event occurs with 
probability 1/2”~1, since the X* and Xj are disjoint substrings of X, and the 
bits of X are chosen independently at random. 


Recall that we assumed the probability H errs on (x, (i, y)) is at most 6. Using 
Proposition 3 and applying a union bound, we get that the error probability of 
IT’ is at most 26, completing the proof of Lemma 1. 


Next, we obtain a lower bound on the one-way communication complexity 
of the modified indexing function. The proof is based on information-theoretic 
arguments. 


Lemma 2. Any one-way deterministic protocol that computes INDn,~ with er- 
ror probability at most € on inputs chosen according to v requires at least (n — 
k)(1 — Ho(e)) bits of communication. 


Proof. Fix any such deterministic protocol, and let A(-) denote the function 
Alice applies on her input in this protocol to determine the message she sends 
to Bob. Bob outputs an answer based on the message from Alice and based 
on his input. Thus, using the random variables A(X), J, and Xjy~%,...,Xy-1, 
Bob is able to predict the random variable Xj with probability of error at most 
g€. This is exactly the scenario captured by a classical result from information 
theory, called Fano’s inequality (cf. [10]), which implies Ho(e) > H(X; | 
A(X), J, Xy_x,-.-,Xy-1). Here H(Z | Y) denotes the conditional Shannon en- 
tropy of the random variable Z given the random variable Y (cf. [10]). By defini- 
tion, the conditional entropy H(Z | Y) equals to >), H(Z | Y = y) - Pr(Y = y), 
where H(Z | Y = y) is the entropy of the conditional distribution of Z given the 
event {Y = y}. Expanding over the random variable J, we thus have: 
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1 << | 
Hi) = —~y 2 H(Xy | A(X), Xy—z,-..,Xs-1, J = J) 
j=k+1 
1 T} 
=-— XO H(X; | A(X), X-k», X4-1). 
j=k+1 





Conditioning one variable on another can only reduce its entropy. Therefore, we 
can lower bound the j-th term on the righthand side by the conditional entropy 
H(X; | A(X), X1,...,Xj-1) (we added the random variables X1, ...,Xj—-x-1 to 
the conditioning). We thus have: 


1 n 
Ha(£) 2 ek >A H(X; | A(X), Xi,..., Xj-1) 


j=k+1 


1 
> =g H(Kegay-+-)Xn | A(X), Xis- +- Xa). 


The last transition follows from the chain rule for entropy. Another applica- 
tion of this rule implies that H(X,41,...,Xn | A(X), X1,..., Xe) = H(X1,..., 
Xn, A(X)) — H(A(X)) — H(X1,..., Xx | A(X)). Since A(X) fully depends on 
X = (Xi,...,Xn), we have 


H(Xi,...,Xn, A(X)) = H(X). 
But, H(X} = n, as X has a uniform distribution on {0,1}”. Since conditioning 
reduces entropy, H(Xj,..., Xx | A(X)) < H(X1,..., Xx) = k. To conclude: 


Hye) > ——- (n-H(A(X)) — 4). 


Therefore, H(A(X)) > (n—k)(1— Ho(e)). Since H(A(X)) is always a lower bound 
on the length of A(X), the lemma follows. 


The proof of Theorem 1 now follows from Lemma 2, Lemma 1, and Yao’s lemma 
[25]. 


For the case when m < O(log n), we have the following lower bound. 


Theorem 2. If n > 2"/3-m, then any 6-error randomized sketching algorithm 
for the pattern matching problem PMy,.m requires a sketch of size at least gm/3—1, 


(1 — H2(6)). 


Proof. The lower bound follows from a reduction from the indexing problem, 
IND,;. The reduction works as follows. Let m be such that t = 2/31) and 
let n > 2/3. m. Given her input x € {0,1}*, Alice maps it first into a string 
x’ € {0,1,$}"/%; $ is a special symbol, which we call a “marker”. Let i1,..., 7% 
be the positions in which x has a 1. Then x’ = 7,$i9$...7,$$...8 where each 
integer 7; is written in binary using logt bits and the trailing $’s are used to 
make sure the string is of length n/3. Bob maps his input j € ft] into the string 
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y’ = j$ in {0, 1, $}”/%. Note that since a marker can match only a marker, = I 
if and only if the pattern y’ matches x’. 

In order to complete the reduction, we need to describe a mapping ¢ from 
strings over the ternary alphabet into (longer) strings over a binary alphabet, 
so that a pattern y’ matches a substring of x’ over the ternary alphabet if and 
only if ¢(y’) matches a substring of ¢(x’). ¢ could be, for example, the following 
mapping: 0 maps to 010, 1 maps to 101, and $ maps to 111. 


4 Approximate Pattern Matching 


In this section we give a nearly-tight sketching lower bound for approximate 
pattern matching. Once again we use the lower bound for the indexing function 
to obtain our lower bound. Recall that we denote by APMn m,- the problem in 
which Alice gets a string x € {0,1}”, Bob gets a string y € {0,1}™, and their goal 
is to determine whether there exists an index i € [n] such that x{t,i+m-—1]=y, 
or whether for all 7 € [n], HD(x{i,2 +m — 1], y) > em, assuming that one of the 
two holds. 


Theorem 3. If n < 29“), then for any constant 0 < € < 1/8, any randomized 
sketching algorithm for APMn,m,e requires a sketch of size 2Q(2). 


Proof. The proof works by a reduction from the indexing function IND:; we 
assume, for simplicity, m divides n, and let t = n/m. We first need to fix a 
collection 21,...,Z2¢ of 2t binary strings in {0,1} with the following property: 
for any m/2 < s < m, and for any i,j, the prefix ofz; of length s and the suffix 
of zj of length s have a large Hamming distance, namely, HD(z,[1, s], z;[m — s + 
1,m]) > ©. A simple probabilistic argument can show that such a collection 
exists as long as t < 27, for some constant 0 < y < 1 ?. We will call the last t 
strings in the collection also 01,...0¢. 

Suppose JT is a sketching algorithm for APMn,m,e. We use it to construct a 
sketching algorithm IZ’ for IND;. Given her indexing input x € {0,1}*, Alice 
maps it into a string u € {0,1}”, which is formed by concatenating t strings of 
length m each. The j-th string is z; if x; = 0 and it is o; if x; = 1. Bob maps 
his input 2 € [¢] into the string v = o;. 

Alice and Bob now run the algorithm J on the inputs u and v. Bob decides 
that x; = 1 if and only if it is determined that v approximately matches u. 

If x; = 1, then v = 0; 1s a substring of u, and therefore the algorithm will 
be correct with high probability. If x; = 0, then v is not one of the strings 
constituting u. We need to use now the property of the collection z1,...,Z2¢ to 
show that no substring of u of length m has small Hamming distance from v. 

Let ut, ...,u* be the strings (which are taken from z1,...,Za¢) that constitute 
u. Suppose, to the contradiction, u has a substring œ of length m such that 


? Note that the corresponding Hamming distance is the summation of indicators for 
the events (z:)e = (2;)e+m—s taken over £ = 1,...,8, and even if į = j, at least 
s/2 > m/2 of them are independent. 
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HD(a,v) < em < m/8. The prefix of œ overlaps some u’ and its suffix overlaps 
uti, Call the overlapping prefix a; and the overlapping suffix a2. At least one 
of a1,@2 has to be of size at least m/2. Suppose, for example, it is œ, and let 
s = |a |. Let vı be the prefix of v of length s. Since the total Hamming distance 
between qa and v is at most m/8, also the Hamming distance between a; and vı 
is at most m/8. But that implies that the Hamming distance between the last s 
bits of u’ and the first s bits of v is at most m/8 even though u? Æ v. This is a 
contradiction to the property of the collection z1,...,Z2¢. We conclude that JT’ 
is correct with high probability also when x; = 0. 
The lower bound now follows from the §2(¢) lower bound for indexing. 


Theorem 4. For any constant £ > 0, there is a randomized sketching algorithm 
for APMnm,e with sketch size O(2e—' log n). 


Proof. We may assume that the text size is at most n’ = (1+¢€)m, because Alice 
can divide the text x into substrings of length (1 + £)m having an overlap of m 
bits between successive substrings, and then Alice and Bob apply an O( z logn) 
sketch independently for of these substrings. If the pattern matches the text, 
then at least one of these substrings must contain that pattern. If the text does 
not contain the pattern, then it suffices to have the algorithm err with probability 
at most 1/n? on each of the =- substrings. 

If the text size is n < (1+ ¢€)m, Alice simply computes O(logn) random 
inner products >> j Xj7j( mod 2) à la Kushilevitz, Ostrovsky and Rabani [19], and 
sends them to Bob. These inner products are tuned to determine whether the 
Hamming distance is at most ¢m/2 or whether it is at least em, namely, each 
bit r; is chosen independently to be 1 with probability 1/(2em) and O with 
probability 1 — 1/(2em). Since each inner product results in a single bit, the 
sketch size is clearly O(log n). 

It remains to show how Bob uses the results of these O(log n) inner products 
to determine, with high probability, whether xfi, i +m — 1] = y. Notice that in 
each inner product, 


Pr[r; = 0 for all j < i and for all j > i + m} = (1 —1/(2em))*™ = (1). 


That is, with probability at least some constant, any single inner product with 
x is actually also a random inner product with xļi, i + m — 1], and hence can be 
used by Bob to estimate whether the Hamming distance HD(xfi, i + m — 1], y} is 
at most em/2 or at least em. The details of this estimate are exactly as in [19]; 
in short, in the latter case the probability that the inner product turns out to be 
1 is higher additively by a constant than in the former. By a standard Chernoff 
bound, with high probability (say at least 1 — 1/n3), there are some (logn) 
inner products that Bob can use to estimate HD(x|i,7 + m — l],y). The proof 
now follows by a union bound over the n’ < n possible values of i. 


Remark. The above sketching algorithm is actually stronger than claimed in 
the theorem, as it determines, with high probability, whether there exists an 
index 7 € [n] such that HD(x[i,2 + m — 1], y) < em/2, or whether for all z € fn], 
HD(x[i,2 +m — 1], y) > em, assuming that one of the two holds. 
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Abstract. In this paper, we study two questions related to the problem of testing 
whether a function is close to a homomorphism. For two finite groups G, H (not 
necessarily Abelian), an arbitrary map f : G — H, and a parameter 0 < € < 1, 
say that f is e-close to a homomorphism if there is some homomorphism g such 
that g and f differ on at most e|G| elements of G, and say that f is €-far otherwise. 
For a given f and €, a homomorphism tester should distinguish whether f is a 
homomorphism, or if f is €-far from a homomorphism. When G is Abelian, it 
was known that the test which picks O(1/e) random pairs x,y and tests that 
f(x) + f(y) = f(x + y) gives a homomorphism tester. Our first result shows 
that such a test works for all groups G. 

Next, we consider functions that are close to their self-convolutions. Let A = 
{ag|g € G} be a distribution on G. The self-convolution of A, A’ = {ag|g € 
G}, is defined by a, = J y seGiyz—2 2y%z: It is known that A = A’ exactly 
when A is the uniform distribution over a subgroup of G. We show that there is 
a sense in which this characterization is robust — that is, if A is close in statistical 
distance to A’, then A must be close to uniform over some subgroup of G. 


1 Introduction 


In this paper, we focus on two questions that are related to the problem of testing 
whether a function is close to ahomomorphism. 

For two finite groups G, H (not necessarily Abelian), an arbitrary map f : G - H, 
and a parameter 0 < e < 1, saythat f is e-close to a homomorphism if there is some 
homomorphism g suchthat g and f differ on at most €|G| elements of G. Define ô, the 
probability of group law failure, by 


1 — 8 = Pr[f(z) x f(y) = f(a x y)]. 


Define 7 such that 7 is theminimum e€ forwhich f is e-close to ahomomorphism. In 
[4], it was shown that over Abelian groups, there is aconstant ôo, such thatif ô < do, 
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then the one can upper bound 7 in terms of a function of 6 that is independent of |G}. 
This yields a homomorphism tester with query complexity that depends (polynomially) 
on 1/e, but is independent of |G]. In particular, the writeup in [4] contains an improved 
argument by Coppersmith [5], which shows that d9 < 2/9 suffices, and that 7 is upper 
bounded by the smaller root of z(1 — x) = 6 (yielding a homomorphism tester with 
query complexity linear in I/e). Furthermore, the bound on ĝo was shown to be tight 
for general groups [5]. 

Our first result is to give a relationship between the probability of group law fail- 
ure and the closeness to being a homomorphism that applies to general (non-Abelian) 
groups. We show that for ôo < 2/9, then f is t-close to a homomorphism where 
T = (3 — V9 — 246)/12 < 5/2 is the smaller root of 3x — 6x? = ő. The condi- 
tion on 6, and the bound on 7 as a function of 6, are shown to be tight, and the latter 
improves that of [4]. 

Next, consider the following question about distributions that are close to their self- 
convolutions: Let A = {a,|g E€ G} be a distribution on group G. The convolution of 
distributions A, B is 


CABi > aby 


y,zEG; yz=2 


Let A’ be the self-convolution of A, A * A, i.e.af, = D e ayaz. Itis known 
that A = A’ exactly when A is the uniform distribution over a subgroup of G. The 
question considered here is: when is A close to A’? In particular, if dist(A, A’) = 
5 reg |@z — @,| < € for small enough e, what can be said about A? We show that A 
must be close to the uniform distribution over a subgroup of G, that is, for a distribution 
A over a group G, if dist(A, A * A) < e < 0.0182, then there is a subgroup H of G 
such that dist( A, Uz) < 5e, where Uy is the uniform distribution over H. On the other 
hand, we give an example of a distribution A such that dzst(A, A * A) = .1504, but A 
is not close to uniform on any subgroup of the domain. 

A weaker version of this result, with a somewhat more complicated proof, was used 
in the original proof of the homomorphism testing result in [4]. The earlier result was 
never published since the simpler and more efficient proof from [5] was substituted. 
Instead, a separate writeup of weaker versions of both of the results in this paper, by the 
current set of authors, was promised in [4]. This paper is the belated fulfillment of that 
promise, though the earlier results have been strengthened in the intervening time. 

To give a hint of why one might consider the question on convolutions of distribu- 
tions when investigating homomorphism testing, consider the distribution Af achieved 
by picking x uniformly from G and outputting f(a). It is easy to see that the error 
probability ô in the homomorphism test is at least dist(As, Ay * Ay). Unfortunately, 
this last relationship is not in the useful direction. In Remark 2 of Section 3, we 
present a relationship between homomorphism testing and distributions close to their 
self-convolution. 


Related Work: The homomorphism testing results can be improved in some cases: We 
have mentioned that dg < 2/9 is optimal over general Abelian groups [5]. However, 
using Fourier techniques, Bellare et. al. [1] have shown that for groups of the form 
(Z/2)", ôo < 45/128 suffices. 
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Several works have shown methods of reducing the number of random bits required 
by the homomorphism tests. That is, in the natural implementation of the homomor- 
phism test, 2 log |G| random bits per trial are used to pick z, y. The results of [7,6,3,8] 
have shown that fewer random bits are sufficient for implementing the homomorphism 
tests. The recent work of [8] gives a homomorphism test for general (non-Abelian) 
groups that uses only (1 + 0(1)) log, |G| random bits. Given a Cayley graph that is 
an expander with Se second eigenvalue ~y, and for the analogous definitions of 


ô, T, they show that for -=— < 1, T is upper bounded by 48/(1 — 7). 


2 Non-Abelian Homomorphism Testing 


In this section, we show that the homomorphism test of [4] works over non-Abelian 
groups as well. As in the Introduction, we define 6, the probability of group law failure, 
by 

1 — ô Preys (yy) Hla xy) 


L,Y 
We prove the following: 


Theorem 1. Jf 6 < 2/9 then f is T-close to a homomorphism, where T = [3 — 
V9 — 246]/12 < 6/2 is the smaller root of 3x — 6x? = 6. 


The rest of this section is devoted to proving the theorem, and showing that the 
parameters of the theorem are tight. 

Pry (x) is the probability of x when x, y are independently selected from G with 
the uniform random distribution. 

Given two finite groups G, H (not necessarily Abelian), and given an arbitrary map 
f : G — H (not necessarily a homomorphism) we will construct a map g : G > H, 
which (under certain conditions on f) will be a group homomorphism and will agree 
with f on a large fraction of its domain G. 

Given f : G — H, with associated 6 < 2/9, we define g : G — H by 


g(a) = majorityzeg |f (a x x) x f(z)~"). 


That is, we evaluate the bracketed expression for each x € G, and let g(a) be the value 
most often attained. Define €a, € and T: 


1 — €a = Pr [f(a x x)f(x)~* = g(a) 
€ = max €a 
1—7 = Pr[f(z) = g(x)] 
Lemma 1. If < 2/9 then €a < ê where ê is the smaller root of x — x? = ô. 


Proof: Fora € G, define 
pPa= Pr |f(ax2) x f(z)" =f(@xy) xf)" |. 


x,yeEG 


By rearranging, we have 
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Pa = Prayec [f(a x y) x fla x z) = f(y)! x f(x) 
> Pri yea [f(a xy) x flax x)= f(y"! x x) A f(y)? x f(z) 
= fy" x x)]. 
Each of the latter two equations is a random instance of the test equation f(u) x f(v) ze 


f(u x v), or equivalently, f(u)~' x f(u x v) = f(v), so each holds with probability 
1 — 6, and, by the union bound, they both hold simultaneously with probability at least 
1 — 26. So we have 

Pa > 1 — 26 > 5/9. 


If we partition G into blocks 
Ba z = {x € G| f(a x z) x f(t! = z} 


with relative sizes ba,z = |Ba,z|/|G|, then 


2 
Da = ` OF < max baz 


z 


so that maxz (ba, z) > 5/9, and g(a) = argmazz(ba,z) is well defined. By definition, 
l — €a = MaX; (ba z) > 5/9. Since 1 — €a > 1/2, we also have 


Pa <(1 — €q)? +e? 


1 — 28 < 1 — 2ea + 2e? 
eae Peas 


and since €a < 1/2, we conclude that €a < ê, the smaller root of x — r? = 6.0 


Corollary 1. If < 2/9 then €a < 1/3 ande < 1/3. 
Lemma 2. If < 2/9 then g is a homomorphism. 


Proof: 

IDENTITY: g(1) = 1. Immediate since each value x gives f (1 x x) x f(x)? =1. 

INVERSE: g(a71) = g(a)~!. There is a one-one correspondence between z satis- 
fying f(a x 2) x f(x)~} = g(a) and y satisfying f(a~! x y) x f(v)! = g(a), 
namely y =a X z. 

PRODUCT: g(a) x g(b) = g(a x b). Each of the following three equations holds 
with probability at least 1 — € > 2/3 on random choice of y: 


gla) = fla x y) x f(y)! 


g(b) = f(y) x f(b" xy)? 
gla x b) = f(a x y) x f(b’ xy) 
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(In the definition of g, we substitute y = x in the first equation, and y = b x x in the 
second and third.) By the union bound, all three equations hold simultaneously with 
probability at least 1 — 3e > Q; that is, there is at least one value of y satisfying all three 
equations. Substituting one such value of y and combining these three equations, we 
conclude g(a) x g(b) = g(a x b), as desired. O 


Lemma 3. T< +€. 


Proof: 


T = Pra [f (a) # g(a)] 
< Praz [f (a) # f(a x x) x f(£)7!] + Prax [g(a) # f(a x 2) x f(£)7!] 
< 6+ Averageg(€a) < +€. 


OJ 
Lemma 4. € > 2(r — 7°). 


Proof: 


Pr [f(e) x fy) # gfe x y™)] 


is the average value of €a (over random choices of a), and so is bounded by e. This 
group law failure will hold at least if either of these two mutually exclusive events 
occurs, since g is a homomorphism: 


— f(x) =9(z) A fly) Fay); 
— f(x) # g(x) A f(y) = gly). 


By the independence of x, y, each of the two events has probability 7(1 — 7). So 
e>7(1—7)+(1—7)t =2(7 - 7”). 
O 


Corollary 2. If ô < 2/9 then t < 3=¥8 < 0.2114. 


Proof: Ifô < 2/9 then e < 1/3, and Lemma 4 implies either T < 3—v3 < 0.2114 or 
T > 3ty3 > 0.7886. The latter is inconsistent with r < 6 + € < 5/9 (Lemma 3). O 


Lemma 5. If 5 < 2/9 then 8 > 3r — 6r?. 


Proof: Since g is a homomorphism, the inequality f(x) x f(y) # f(x x y) (which 
has probability 6) will hold in at least the following three mutually exclusive events: 


- f(z) = g(x) A f(y) = g(y) A f(z x y) £ g(x x y); 
- f(x) = g(z) A f(y) £ gly) A f(z x y) = g(x x y); 
- f(x) Æ g(x) A f(y) = gly) ^ f(z x y) = g(a x y). 


oO 


Lemma 6. If ô < 2/9 then T is bounded by the smaller root of 3x — 6r? = 6. 
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Proof: Combine Corollary 2 (T < 0.2114) with Lemma 5 (6 > 37 — 67°). O 
This finishes the proof of Theorem 1. 


Example I. The bound 6 < 2/9 is tight. The following example has 6 = 2/9 and 
€ = 1/3, but r = 1 — 1/3*~? is arbitrarily close to 1. 


S223 a7)" 
f(3€+d) = 4,0 < £ < 3" ',de {-1,0,1} 
More details are given in the full version [2]. 
Example 2. The bound 37 — 67? < 6 is tight. Choose 7’ with 0 < 7’ < 1/3, choose N 
an arbitrarily large odd positive integer, and define f : Z/N — Z/2 by 


f(g)=1ler <a <2. 


More details are given in the full version [2]. 


3  Convolutions of Distributions 


In this section, we show that for a distribution A over a finite group G, if |A—Ax*A] < € 
then A is 6-close to the uniform distribution over a subgroup of G. 

We let capital letters A,B,C denote distributions over group G and subscripted 
uncapitalized letters ay, by denote the probability of a particular element. X, Y, Z, H 
will be subsets of G. 

We let Us denote the uniform distribution on S C G. 

We let dist(A, B) = $|A — B|. Note that distances satisfy the triangle inequality, 
i.e., dist(A, C) < dist(A, B) + dist(B,C). Also it is easy to see that dist( A + B, A x 
C) < dist(B,C). 

It will also be convenient to consider a second kind of convolution, 


C= Áe B, G&G = ` Aybz. 


y,zEG; ry=z 


When we have uniform distributions on subsets of equal size, the two convolutions 
enjoy the following relation, the proof of which is given in the full version [2]: 


Lemma 7. Let X, Y, Z be subsets of a finite group G, with |X| = |Y| = |Z| =n. 
Then 
dist(Ux, Uy * Uz) = dist(Uy, Uz e Ux). 


Remark I. The lemma does not hold for arbitrary distributions, nor for uniform distri- 
butions on subsets of different sizes. 


Overview of Proof: We will embed G in a larger group F = GxZ/N for suitably large 
N, and consider a distribution B induced from A, namely biz j) = Qz /N. This will 
alleviate problems later when we have to round to integers. We show that if B’ = Bx B 
is close to B, then there is a set X C F such that B is close to Ux. We next show that 
X must be close to a subgroup H of F, and further that this subgroup is of the form 
H = H x Z/N. Then B is close to the uniform distribution on H, and A is close to 
the uniform distribution on H. A bootstrap lemma allows us to claim that once A is 
moderately close to Uy, then it is very close. 
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Expanding the Group: Pick N suitably large. Define F = G x Z/N, with elements 
{(z,j): 2 € G,j E€ Z/N} and group law (2, 7)(y,k) = (xy, j + k). The distribution 
B on F is given by A x UZ/n, that is, bis j) = @z(1/N). Defining B’ = B * B and 
A’ = Ax A, it is immediate that dist(B, B’) = dist(A, A’). 


B is Close to Uniform on a Subset: Our first theorem shows that if B’ = B x B is close 
to B, then there is a set X C F such that B is close to Ux. 


Theorem 2. Let F be a finite group. Let B be a distribution on F for which no element 
has probability more than 1/N. Let 1/8 > € > 0 be a constant. If dist(B, B » B) <€ 
then there isa set X C G such that dist(B,Ux) < € where € = 3e + O(1/N). 
Further, dist(Ux,Ux * Ux) < 6e + O(e?) + O(1/N). 


Proof: Let B’ = B x B. In the rest of the proof, relabel the elements such that 
by > bo È bg > ...,1.., 1 corresponds to the element of F with the highest probability 
mass. For given z € G, the N elements 6, 4), k E Z/N, are equal, so we arrange that 
they are contiguous in this ordering. 

Forn > 1, let sum, = D b; be the sum of the n highest probabilities. Also, let 
suw, = >. j=l b; be the sum of the probabilities with respect to B’ of the n most likely 
elements with Oee to B. 


Let dn = sum? + n Y`., b2. Itis not hard to see that sum, > Qn. 


jon ~j’ 


; / 
Claim. A, > sum,,. 


Proof: [of claim] Equivalently, 


oe) Be O Pag a 


j>n TY= Zj 


The right-hand side can be converted to the left-hand side by a carefully selected se- 
quence of “edge swaps”, replacing bbi + byb by bkbk' + by b; when k < I’ and 
k’ < l. Each edge swap does not decrease the sum. Details in the full version [2]. 0 

We will define an X such that dist(B, Ux) is small. Pick 7 with 1/4 < 7 < 3/4; 
later we will specify r = 3/5. Select m with Summ- < T < sum,,. Set h = bm. Set 
p = |1/h|. Let the distribution U assign weight A to the first p elements, and a weight 
1 — ph < h to the (p + I)st element. Let n = p if bp41 > ûp+1, and n = p41 
otherwise. Let X consist of the first elements, so that dist(U, Ux) <h=O(1/N). 
Also define g = bn. 

The distribution B differs from U in three places. Fort < m, b; > û;; define 
B= >_<, (bi — Gi). Fori > n, b; > û;; define ô = dain (bi — ùi). Frm <i<n, 
b; < ù; we have B +6 = do ic, (ti — bi). So dist(B, U) =B+6. 

Form < i < n wehaveg < b; < h, so that b? < (g + h)b; — gh. Similarly for 
i > n we have 0 < b; < g, so that b? < gb;. Recall also that mh + 8 = r + O(1/N) 
and nh = 1 + O(1/N). This enables the following computation: 
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€ > sum, — sum), > summ — Am 
= (mh +) ~ (mh +B)? -mI BR — MDs B 
> (mh + B)(1 — mh — L) — m(g + h) 04, bi + m(n — m)gh — mg X,,,, bi 
= (mh + B)(1 — mh — B) —m(g4+h)(nh — mh — B — ô) 
+mngh — m*gh — mgd 
= (1 — mh — f)(B) — (1 — mh — B)(mg) + mhô + mg — m? gh + O(1/N) 
= (1—7)(8) + mhô + mgB + O(1/N) 
> (1-—7)8+76 — Bå + O(1/N) 
= [(1—17)8 +76] + pasa {l(l — 7)8 — 76]? — [(1 — 7) 6 + 75]?} + O(1/N) 
> [(1— 1) 8 + 78] - yill — 7)2 + rô]? + O(1/N) 
=u- pissy + O(1/N) 
where u = (1—7)6+76.Now B < 7+ O(1/N) and 6 < 1—7 so that u = 
(1 —7)8 +76 < 27(1 —7) is less than the larger root of 
ge 


ee = oe = 
4r(1 — 7) TE ; 


namely 
(1 -—rT)B +76] < 27(1 —7) + O(1/N) < 27(1 —7)[1 + V1 -— €/7(1 —7)] 
(since €/7(1 — T) < (1/8)/(1/4 x 3/4) < 1), so it must be less than the smaller root: 


(1 —7)8 +76] < 27(1 —7)[1 — V1 — e/T(1 —7)] + O(1/N), 


remembering the error term. 
Substituting T = 3/5, we have 


12 | 25 
yan bee nec 
264+ 36 < z n 1 z€ 


By the triangle inequality, 


+ O(1/N) = 5e + O(c?) + O(1/N). 





dist(B,Ux) < dist(B,U) + dist(U,Ux) = 8 +ô + O(1/N). 


One can calculate that ife < 1/8 then 8 + 6 < 3e, establishing the theorem. For later 
use, we also note that if € < 0.0182 then 8 + ò < 2.6e. 
Repeatedly applying the triangle inequality, we can obtain: 


dist(B, Ux * Ux) < dist(B, B » B) + dist(B, B xUx) + dist(B, Ux * Ux) 
<e+2(6 +ô). 


To obtain dist(Ux,Ux * Ux) we could apply the triangle inequality again. Instead, 
we recall that B differs from U, and hence Ux, in three pieces, namely (up to errors 
of order O(1/N)) (before m), —8 — 6 (between m and n), and ô (after n). The 
convolution Uy * Ux is everywhere bounded by n(1/n)* = 1/n. So when we change 
B to Ux and monitor the change in dist(x, Ux * Ux), the first change of 8 is driving 
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us closer to Ux * Ux, while the other two changes of 8 + 6 and 6 might drive us further 
away. The net result is 


dist(Ux,Ux * Ux) < dist(B, Ux * Ux) + ł -8+ (6+ 6) +8] + O001/N) 
< e€ +28 +38 + O(1/N) 
<e+# 1 -4/1 — Be + O(1/N) 
= 6e + O(e7) + O(1/N). 
This establishes the second part of the theorem.0 


B Is Close to Uniform on a Subgroup of F: Next we show that if the uniform distri- 
bution on X is close in distance to its convolution with itself, then X is close to some 
subgroup H of F. 


Theorem 3. Let F be a finite group and X a subset of F. Let T = dist(Ux,Ux * 
Ux) = dist(Ux,Ux eUx). Ift < 1/9 then there is a subgroup H of F with |X\H| + 
|H\X] < 37|X|. 


Proof: Letn = |X|. Let V = Ux e Ux so that 


1 
Ur = 7a lily, 2) : Y, z E X, £y = z}|. 


If e is the identity element of F, we see ve = +, and vz < + forall z € G. 


— n 


We need to establish a triangle inequality on quantities such as (ve — vz). 


Lemma 8. Forz,y € F, the quantities (ve —Vz), (Ve — Vy), (Ve — Vry) are nonnegative 
and satisfy the triangle inequalities: 


(ve — Ur) + (Ve — Vy) 2 (Ve — Vry) (1) 
(Ve — Ve) + (Ve — Vay) = (Ve — Vy) (2) 
(Ve So Vy) tr (Ve a Ug) > (Ve E Vz) (3) 


Proof: Ifv, = kz /n?, with O < k} < n, then there are k, elements z € F such that 
both z and xz are in X; call such z “good elements” for x. There are n — ky elements z 
such that z € X and xz ¢ X; there are n—k, elements z such that z ¢ X and zz € X; 
call the latter two kinds of z “bad elements” for x. The number of bad elements for æ is 
2(n — kz) = 2n? (ve — vz). If z is neither good nor bad for z it is “neutral” for z. 

For each z € F, consider the three elements z, yz, xyz. If all three are in X, then 
we have found good elements for each of z, y, xy. (Namely, z is a good element for 
y and for xy, and yz is a good element for x.) If exactly two are in X, then we have 
found bad elements for exactly two of x, y, xy, and a good element for the other. (For 
example, if z, xyz € X and yz ¢ X, then z is good for xy, z is bad for y, and yz is 
bad for x.) If exactly one of z, yz, xyz is in X, then we have bad elements for exactly 
two of x, y, xy and a neutral one for the other. If z, yz, xyz ¢ X then we have found 
neutral elements for all three. The important point is that the “bad elements” come in 
pairs, each time contributing to two of 2(n — kz), 2(n — ky), 2(n — kzy). Setting 


p= l{z : z bad for ry and y}| 
q = |{z : z bad for xy; yz bad for x}| 
r = |{z: z bad for y; yz bad for z}}, 


we find 
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2(n — kr) =q +r 
2(n — ky) =p+r 
2(n — kry) =p+q. 
This establishes the triangle inequality among 2(n — kz), 2(n — ky), 2(n — key), and 
hence the lemma. 0 
Next we show an “excluded middle” result for V. 


Lemma 9. Let r < 1/9. Forall x € F, either vy < 32 < È or v; > 3 > SJ. 


3T 


Proof: Assume the contrary: for some x € F, 22 < vs < +=32. Choose any y € X. 


If cy € X, use triangle inequality (3) to deduce 


(Ve — Vy) + (Ve — Vry) Z (Ve — Vz) 
|(Ux )y — vy] + |(Ux Jay — Veyl 2 (Ve — vz) > * - (3 -7 =r) = 5. 


If xy ¢ X, use triangle inequality (1) and (Ux). = 0 to deduce 
(Ve — Uz) + (Ve — Vy) > (Ve — Vay) 
(Ve — Vy) + Vey 2 Ve 
(Ox )y — vy| + (Ux )2y — Vry| = vz > 3r, 


Summing over y € X, 


` (Ox )y — vy] + (Ux zy — vzyl|] > n (=) = 3r. 


yEX 

Then use 
dist(Ux, V) = X |(Ux)y — vyl 
yEX 

and 

2dist(Ux,V) = So (Ux )z — vz| > y (Ux )zy — Vey 

zEF yEX 

to deduce 


(1 + 2)dist(Ux,V) > 37 
37 = 3dist(Ux,V) > 37, 
contradicting our hypothesis and establishing the lemma. 0 


The excluded middle gives us a natural candidate for our subgroup H. The follow- 
ing is proved in the full version [2]. 


Lemma 10. Let r < 1/9. Define H = {x € F : v, > +}. Then H is a subgroup 
of F. 
Finally we show that Ê is close to X. The proof is given in the full version [2]. 


Lemma 11. With T < 1/9 and H as above, the symmetric difference between X and 
H satisfies: 
< 





A a 2 
JÊ\XI + [X\H] < |X| < 37|X|. 


i 





This establishes the theorem.D 
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B Is Close to a Subgroup H: We can push these results back to the distributions B: 


Theorem 4. Let F be a finite group. Let B be a distribution on F. Let X C F bea 
subset. Given €, B, ô such that: 


- dist(B, Bx B) <6 
— dist(B,Ux) < 8 + ô (as in Theorem 2); 


then there is a subgroup H of F with dist(B,Uy) < B+ 64 27/(1 — 37). The same 
is true if we replace the first condition with 


dist(B, Be B) <e. 
The proof uses Theorems 2 and 3, and is given in the full version [2]. 


Reverting to Original Group: The group H respects the block structure of F = G x 
Z/N, in the sense that for all x # e € G and j,k € Z/N, Væ j) = Viak) With v 


as defined in Lemma 8, so that (z, j) € Ê & (z,k) € H. Further, one can verify 
that vek) > Væk) SO that if any (x,k) € H with x # e, then the entire block 


{(e,k) : k € Z/N} is in Ê. (Note that if it were the case that there were no (z,k) € H 
with x # e, then H = {e}, which is a subgroup of G.) This implies that H is of the 
form 

H=HxZ/N. 


It is obvious that H is a subgroup of G, and that 
dist(A, Uy) = dist(B, Uz). 
We tie in with Theorem 2. 


Theorem 5. Let A be a distribution on the finite group G. Let dist(A,A* A) < e < 
0.0182. Then there is a subgroup H C G with dist(A, Uy) < 21e. 


Proof: Pass to B and F, with 
dist(B, Bx B) <e. 
A computation shows that with e < 0.0182, we have 
28 +38 < 2 [1-4/1 - Bel + O(1/N) < 5.1 
B+6< 2.6¢. 
From Theorem 2 we have a subset X with 
dist(B,Ux) < B +å < 2.6e. 


Then apply Theorem 4 with 7 = e + 28 +38 < 6.le < 1/9 to find the subgroup H, 
and use the triangle inequality: 

2 
dist(B, Up) < dist(B,Ux) + dist(Ux, Up) <B+6+ — < 2.6€+ 37 < 2le. 
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Reverting to the original distribution, 
dist(A, Uy) = dist(B, Uy) < 21e. 


E 

Once we have bounds on dist(A, A * A) (or dist(A, A e A)) and a subgroup H 
with small dist(A, Uz), we can improve the numerical estimates of dist( A, Uz). The 
following is proved in the full version of the paper [2]: 


Theorem 6. Given a distribution A on G and a subgroup H C G with 
dist(A, A x A) = e < 0.06 
dist( A, Upg) = p < 0.4 


then we can conclude 
dist(A, Up) < 5e. 


Combining the last two results, we have: 


Theorem 7. Let A be a distribution on the finite group G. Let dist( A, A * A) < € < 
0.0182. Then there is a subgroup H C G with dist(A, Uy) < 5e. 


Proof: From Theorem 5 we have such a subgroup H with dist(A,Uy) < 2le < 
0.3822 < 0.4. Since 0.0182 < 0.06, Theorem 6 applies, giving dist(A, Uy) < 56. O 


Example 3. Let G = Z/N with N a large prime integer. Let an = v(240 — |n|) when 
—200 < n < —l or 1 < n < 200 and an = 0 otherwise, where v = 1/55800 is chosen 
to normalize A. Then dist(A, A * A) = 0.1539, but dist(A, Uy) = 1 — O(1/N) for 
any subgroup H of Z/N. 


A gap remains. 


Remark 2. The two notions explored in this paper (homomorphism testing, and dis- 
tributions close to their self-convolution) are related. Given a map (not necessarily a 
homomorphism) f : G — H between two finite groups, we can construct the product 


group 
Gx H={(z,y):2€G,ye H} 


and a distribution A: . 
4 - SUNG! ify = F@) 
(x,y) 0 otherwise 
Then we have the identity 
dist(A, A x A) = Pr{f(2) x f(y) # F(E x y)]. 


If f is close to a homomorphism g, then A is close to the uniform distribution on the 
subgroup {(x, g(z)) : x € G}. Conversely, if f is likely to pass the homomorphism 
test, or equivalently, if A is such that dist(A, A * A) is small enough, then f must be 
close to a homomorphism: The theorem from this section implies that A must be close 
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to a distribution that is uniform over a subgroup S of G x H. One can show that S is 
such that for each a € G,|S{){(a, b) : b € H}| = 1. Then S can be used to define a 
map g : G — H such that g is a homomorphism (this can be seen from the underlying 
group structure of S) and such that g agrees with f on most inputs. 

But the correspondence when dist(A, A * A) is larger is not exact: the map f given 
in Example 1 is not at all close to any homomorphism g on all of G, but there is a 
subgroup H of GxH with dist(A, U,,) = 2/3, namely H = {(34, £) : 0 < £ < 3571}. 
The difference comes because g is required to be a homomorphism on all of G. We 
could relax the requirement, and notice that there is a large subgroup G” of G (namely 
G" = {3¢}, with |G’| = |G|/3) and a homomorphism g’ : G’ — H that agrees with f 
on this subgroup. The subgroup H of G x H is associated with G” and g’. 
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Abstract. We continue the investigation of locally testable codes, i.e., 
error-correcting codes for whom membership of a given word in the code 
can be tested probabilistically by examining it in very few locations. 
We give two general results on local testability: First, motivated by the 
recently proposed notion of robust probabilistically checkable proofs, we 
introduce the notion of robust local testability of codes. We relate this 
notion to a product of codes introduced by Tanner, and show a very 
simple composition lemma for this notion. Next, we show that codes 
built by tensor products can be tested robustly and somewhat locally, 
by applying a variant of a test and proof technique introduced by Raz 
and Safra in the context of testing low-degree multivariate polynomials 
(which are a special case of tensor codes). 

Combining these two results gives us a generic construction of codes of 
inverse polynomial rate, that are testable with poly-logarithmically many 
queries. We note these locally testable tensor codes can be obtained from 
any linear error correcting code with good distance. Previous results 
on local testability, albeit much stronger quantitatively, rely heavily on 
algebraic properties of the underlying codes. 


1 Introduction 


Locally testable codes (LTCs) are error-correcting codes that admit highly ef- 
ficient probabilistic tests of membership. Specifically, an LTC has a tester that 
makes a small number of oracle accesses into an oracle representing a given word 
w, accepts if w is a codeword, and rejects with constant probability if w is far 
from every codeword. LTCs are combinatorial counterparts of probabilistically 
checkable proofs (PCPs), and were defined in [18,24,2], and their study was 
revived in [20]. 

Constructions of locally testable codes typically come in two stages. The 
first stage is algebraic and gives local tests for algebraic codes, usually based on 
multivariate polynomials. This is based on a rich collection ofresults on “linearity 
testing” or “low-degree testing” [1,3—9,13,14,16—-18,20,22,24]. This first stage 


K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 286—297, 2004. 
© Springer-Verlag Berlin Heidelberg 2004 


Robust Locally Testable Codes and Products of Codes 287 


either yielded codes of poor rate (mapping & information symbols to codewords 
of length exp(k)) as in [14], or yielded codes over large alphabets as in [24]. 
To reduce the alphabet size, a second stage of “composition” is then applied. 
In particular, this is done in [20,13,11] to get code mapping k information 
bits to codewords of length k!+°™), over the binary alphabet. This composition 
follows the lines of PCP composition introduced in [4], but turns out to be fairly 
complicated, and in most cases, even more intricate than PCP composition. The 
one exception is in [20, Section 3], where the composition is simple, but based on 
very specific properties of the codes used. Thus while the resulting constructions 
are surprisingly strong, the proof techniques are somewhat complex. 

In this paper, we search for simple and general results related to local testing. 
A generic (non-algebraic) analysis of low-degree tests appears in [19], and a 
similar approach to PCPs appears in [15]. Specifically, we search for generic 
(non-algebraic) ways of getting codes, possibly over large alphabets, that can be 
tested by relatively local tests, as a substitute for algebraic ways. And we look 
for simpler composition lemmas. We make some progress in both directions. We 
show that the “tensor product” operation, a classical operation that takes two 
codes and produces a new one, when applied to linear codes gives codes that 
are somewhat locally testable (See Theorem 1). To simplify the second stage, we 
strengthen the notion of local testability to a “robust” one. This step is motivated 
by an analogous step taken for PCPs in [11], but is naturally formulated in 
our case using the “Tanner Product” for codes [26]. Roughly speaking, a “big” 
Tanner Product code of block-length n is defined by a “small” code of block- 
length n’ = o(n) and a collection of subsets S1,..., Sm C [n], each of size n’. A 
word is in the big code if and only if its projection to every subset S; is a word 
of the small code. Tanner Product codes have a natural local test associated 
with them: to test if a word w is a codeword of the big code, pick a random 
subset S; and verify that w restricted to S; is a codeword of the small code. The 
normal soundness condition would expect that if w is far from every codeword, 
then for a constant fraction of such restrictions, w restricted to 5S; is not a 
codeword of the small code. Now the notion of robust soundness strengthens 
this condition further by expecting that if w is far from every codeword, then 
many (or most) projections actually lead to words that are far from codewords of 
the small code. In other words, a code is robust if global distance (from the large 
code) translates into (average) local distance (from the small code). A simple, 
yet crucial observation is that robust codes compose naturally. Namely, if the 
small code is itself locally testable by a robust test (with respect to a tiny code, 
of block-length o(n’)), then distance from the large code (of block-length n) 
translates to distance from the tiny code, thus reducing query complexity while 
maintaining soundness. By viewing a tensor product as a robust Tanner product 
code, we show that a (log N/ log log N)-wise tensor product of any linear code 
of length n = polylog N and relative distance 1 — ee yw=1- +, which yields 
a code of length N and polynomial rate, is testable with poly(log N) queries 
(Theorem 2). Once again, while stronger theorems than the above have been 
known since [6], the generic nature of the result above might shed further light 
on the notion of local testability. 
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Organization. We give formal definitions and mention our main theorems in 
Section 2. In Section 3 we analyze the basic tester for tensor product codes. 
Finally in Section 4 we describe our composition and analyze some tests based 
on our composition lemma. 


2 Definitions and Main Results 


Throughout this paper 2 will denote a finite alphabet, and in fact a finite field. 
For positive integer n, let [n] denote the set {1,...,n}. For a sequence x € X” 
and i € [n], we will let x; denote the ith element of the sequence. The Hamming 
distance between strings z,y € X”, denoted A(z, y), is the number of i € [n] 
such that x; # y;. The relative distance between x,y E€ X”, denoted (x,y), is 
the ratio A(z, y)/n. 

A code C of length n over 4 is a subset of £”. Elements of C are referred 
to as codewords. When % is a field, one may think of £” as a vector space. If 
C is a linear subspace of the vector space X”, then C is called a linear code. 
The crucial parameters of a code, in addition to its length and the alphabet, 
are its dimension (or information length) and its distance, given by A(C) = 
minggyec{A(z, y)}. A linear code of dimension k, length n, distance d over the 
alphabet £ is denoted an [n,k,d]s code. For a word r € £” and a code C, 
we let dc(r) = minzec{d(r,xz)}. We say r is 6’-proximate to C (6’-far from C, 
respectively) if dc(r) > 6’ (dc(r) > 6’, respectively). 

Throughout this paper, we will be working with infinite families of codes, 
where their performance will be measured as a function of their length. 


Definition 1 (Tester). A tester T with query complexity q(-) is a probabilistic 
oracle machine that when given oracle access to a string r € E”, makes q(n) 
queries to the oracle for r and returns an accept/reject verdict. We say that T 
tests a code C if whenever r € C, T accepts with probability one; and when 
r gC, the tester rejects with probability at least dc(r)/2. A code C is said to be 
locally testable with q(n) queries if there is a tester for C with query complexity 
q(n). 

When referring to oracles representing vectors in 4”, we emphasize the 
queries by denoting the response of the ith query by rfi], as opposed to ri. 
Through this paper we consider only non-adaptive testers, 1.e., testers that use 
their internal randomness R to generate q queries i;,...,ig E€ [n] and a predicate 
P : 34 — {0,1} and accept iff P(r[t1],...,r[tg]) = 1. 

Our next definition is based on the notion of Robust PCP verifiers introduced 
by [11]. We need some terminology first. 

Note that a tester T has two inputs: an oracle for a received vector r, and a 
random string s. On input the string s the tester generates queries i1,...,%9 € [n] 
and fixes circuit C = Cs and accepts if C(r[ii],...,r{t,]) = 1. For oracle r and 
random string s, define the robustness of the tester T on r,s, denoted pt (r, s), 


to be the minimum, over strings x satisfying C(x) = 1, of relative distance 


of (rlii],...,r[ig]} from z. We refer to the quantity p7 (r)  B,[p7(r,s)] as 


the expected robustness of T on r. When T is clear from context, we skip the 
superscript. 
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Definition 2 (Robust Tester). A tester T is said to be c-robust for a code C 
if for every r € C, the tester accepts w.p. one, and for every r € X”, dc(r) < 


c-pt (r). 


Having a robust tester for a code C implies the existence of a tester for C, 
as illustrated by the following proposition. It’s proof appears in the full version 
of the paper [12]. 


Proposition 1. Jf a code C has a c-robust tester T for C making q queries, 
then it is locally testable with O(c: q) queries. 


The main results of this paper focus on robust local testability of certain 
codes. For the first result, we need to describe the tensor product of codes. 


Tensor Products and Local Tests. Recall that an [n, k, d] s linear code C may be 
represented by a k x n matrix M over X (so that C = {aM |x € X*}). Such a 
matrix M is called a generator of C. Given an [n1, kı, dıl} code Cı with gener- 
ator Mı and an [n2,k2,de]x code Ca with generator M2, their tensor product 
(cf. [21], [25, Lecture 6, Section 2.4]), denoted C1 @ Cy C 3"2*™1, is the code 
whose codewords may be viewed as ng x nı matrices given explicitly by the set 
{Mi XM,|X e X*2**1}. It is well-known that Cy Q C3 is an [n1N2, kiko, dido] 5 
code. 

Tensor product codes are interesting to us in that they are a generic construc- 
tion of codes with “non-trivially” local redundancy. To elaborate, every linear 
code of dimension k does have redundancies of size O(k), i.e., there exist subsets 
of t = O(k) coordinates where the code does not take all possible ©” possible 
values. But such redundancies are not useful for constructing local tests; and 
unfortunately generic codes of length n and dimension k may not have any re- 
dundancies of length o(k). However, tensor product codes are different in that 
the tensor product of an |n, k, d]5 code C with itself leads to a code of dimension 
k? which is much larger than the size of redundancies which are O(k)-long, as 
asserted by the following proposition. 


Proposition 2. A matrix r € X™?*™ is a codeword of Ci ® C2 if and only if 
every row is a codeword of C and every column is a codeword of C4. 


In addition to being non-trivially local, the constraints enumerated above are 
also redundant, in that it suffices to insist that all columns are codewords of Ch 
and only kz (prespecified) rows are codewords of C1. Thus the insistence that 
other rows ought to be codewords of Cy; is redundant, and leads to the hope 
that the tests may be somewhat robust. Indeed we may hope that the following 
might be a robust test for C1 ® Co. 


Product Tester: Pick b € {1,2} at random and i € [ng] at random. 
Verify that r with bth coordinate restricted to 7 is a codeword of C3—b. 


While it is possible to show that the above is a reasonable tester for C1 @ C2, 
it remains open if the above is a robust tester for C1 ® C2. (Note that the query 
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complexity of the test is max{n1, n2}, which is quite high. However if the test 
were robust, there would be ways of reducing this query complexity in many 
cases, as we will see later.) 

Instead, we consider higher products of codes, and give a tester based on an 
idea from the work of Raz and Safra [23]. Specifically, we let C™ denote the code 
C ®---@C. We consider the following test for this code: 

‘nero soem! 


m 


m-Product Tester: Pick b € [m] and i € [n] independently and uni- 
formly at random. Verify that r with bth coordinate restricted to i is a 
codeword of C™7~?, 


Note that this tester makes N!~* queries to test a code of length N = n”. So 
its query complexity gets worse as m increases. However, we are only interested 
in the performance of the test for small m (specifically m = 3,4). We show that 
the test is a robust tester for C™ for every m > 3. Specifically, we show 


Theorem 1. For a positive integer m and{n, k, d]-code C, such that (4=+)" > 


z, m-Product Tester is 21$-robust for C™. 


This theorem is proven in Section 3. Note that the robustness is a constant, 
and the theorem only needs the fractional distance of C to be sufficiently large 
as a function of m. In particular a fractional distance of 1 — 5+, suffices. Note 
that such a restriction is needed even to get the fractional distance of C™ to be 
constant. 

The tester however makes a lot of queries, and this might seem to make this 
result uninteresting (and indeed one doesn’t have to work so hard to get a non- 
robust tester with such query complexity). However, as we note next, the query 
complexity of robust testers can be reduced significantly under some circum- 
stances. To describe this we need to revisit a construction of codes introduced 
by Tanner [26]. 


Tanner Products and Robust Testing. For integers (n,m,t) an (n,m, t)-ordered 
bipartite graph is given by n left vertices [n], and m right vertices, where each 
right vertex has degree t and the neighborhood of a right vertex j € [m] is 
ordered and given by asequence £; = (¢;1,..., 2,4) with l; € fn]. 

A Tanner Product Code (TPC), is specified by an [n,m,t] ordered bipar- 
tite graph G and a code Csma C &*. The product code, denoted TPC(G = 
{big xcitm s Comal) S22" 48 the set 

{r E€ ay | rle; 2 CERN a hs) S Canal Vj = [m]}. 

Notice that the Tanner Product naturally suggests a test for a code. “Pick a 
random right vertex j € [m] and verify that rle, E Cyman-” Associating this test 
with such a pair (G,Csmau), we say that the pair is c-robust if the associated 
test is a c-robust tester for TPC(G, Cgman). 

The importance of this representation of tests comes from the composabil- 
ity of robust tests coming from Tanner Product Codes. Suppose (G,C sma) is 
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c-robust and Csmail is itself a Tanner Product Code, TPC(G’, Ceman’) where 
G’ is an (d,m’,t’)-ordered bipartite graph and (G’,Cyman’) is ¢c’-robust. Then 
TPC(G, Csman) has an c-c’-robust tester that makes only t’ queries. (This fact 
is completely straightforward and proven in Lemma 3.) 

This composition is especially useful in the context of tensor product codes. 
For instance, the tester for C4 is of the form (G,C%), while C? has a robust 
tester of the form (G’,C?). Putting them together gives a tester for C+, where 
the tests verify appropriate projections are codewords of C*. The test itself is 
not surprising, however the ease with which the analysis follows is nice. (See 
Lemma 4.) Now the generality of the tensor product tester comes in handy as 
we let C itself be C’? to see that we are now testing C’ where tests verify some 
projections are codewords of C’4. Again composition allows us to reduce this to 
a C’*-test. Carrying on this way we see that we can test any code of the form 
C2 by verifying certain projections are codewords of C?. This leads to a simple 
proof of the following theorem about the testability of tensor product codes. 


Theorem 2. Let {Ci}; be any infinite family of codes with C; a [ni, ki, dil», 
code, with ni = p(k) for some polynomial p{-). Further, let t; be a sequence of 


integers such that m; = 2° satisfies d,/n; >1— = Then the sequence of codes 


_¢ 
2 


{Ci = Ci}, is a sequence of codes of inverse polynomial rate and constant 
relative distance that is locally testable with polylogarithmic number of queries. 


This theorem is proven in Section 4. We remark that it is possible to get 
code families C; such as above using Reed-Solomon codes, as well as algebraic- 
geometric codes. 


3 Testing Tensor Product Codes 


Recall that in this section we wish to prove Theorem 1. We first reformulate this 
theorem in the language of Tanner products. 

Let G}, denote the graph that corresponds to the tests of C™ by the m- 
Product Tester, where C C X”. Namely Gy, has n™ left vertices labelled by 
elements of [n]™. It has m -n right vertices labelled (b,i) with b € [m] and 
i € [n]. Vertex (b,i) is adjacent to all vertices (%1,...,%m) such that i, = i. 
The statement of Theorem 1 is equivalent to the statement that (G?,,C™~1) 
is 2!§-robust, provided (4=)” > £. The completeness of the theorem follows 
from Proposition 2, which implies C™ = TPC(G”,,C™~+). For the soundness, 
we first introduce some notation. 

Consider the code C1 ®--- @ Cm, where C; = [n;, ki, dj] code. Notice 
that codewords of this code lie in 1™:*"'*"~, The coordinates of strings in 
mix X"m are themselves m-dimensional vectors over the integers (from [n1] x 

-X [Nm]). For re Y™*"*"™ and i1,...,¢m with i; € [nj], let r[ti,...,%m| 
denote the (i1,...,%,)-th coordinate of r. For b € [m], and i € [no], let roi E€ 
D/MIX XNG—-LXN6+1%"""XNm He the vector obtained by projecting r to coordinates 
whose bth coordinate is i, i.e., re ¿fir ..-,îm-=1] = 7[t1,--- 5 2-1, t, ib, -p tm—1]. 

The following simple property about tensor product codes will be needed in 
our proof. It’s proof appears in the full version of the paper [12]. 
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Proposition 3. For 6 € {1,...,m} let C, bean [nv, kb, dels code, and let Iy 
be a set of cardinality at least na — dy +1. Let C; be the code obtained by the 
projection of Cy to Ia. Then every codeword œ of Ci & -8 Clan can be extended 
to a codeword c of Cy ®-:-® Cm. 


Recall that the m-Product tester picks a random b € [m] and 7 € [n] and 
verifies that ra; € C™~1. The robustness of this tester for oracle r on random 
string s = (b, i) is given by p(r, (b, i)) = dgm-1(79,;), and its expected robustness 
is given by p(r) = Es,i{dem-1(rp,:)]|. We wish to show for every r that dgm(r) < 
2'6 . p(r). 

We start by first getting a crude upper bound on the proximity of r to C™ 
and then we use the crude bound to get a tighter relationship. To get the crude 
bound, we first partition the random strings into two classes: those for which 
the robustness p(r, (b,7)) is large, and those for which it is small. More precisely, 
forr € ©” and a threshold 7 € [0,1], define the t-soundness-error of r to 
be the probability that dem-1{r,i) > T, when b € [m] and z € |n} are chosen 
uniformly and independently. Note that the ,/p-soundness error of r is at most 
Vpfor p = p(r). We start by showing that r is O(7 + €)-close (and thus also 
O(,/p)-close) to some codeword of C™. 


Lemma 1. Ifthe t-soundness-error of r is € for T +2e < 4 -(S*)", then 
Som(r) <16- (2) -(r +6). 


Proof. For every i € [n] and b € [m], fix cy; to be a closest codeword from C™~! 
to rbi. We follow the proof outline of Raz & Safra [23] which when adapted to 
our context goes as follows: (1) Given a vector r and an assignment of codewords 
Cpi E C™—!, we define an “inconsistency” graph G. (Note that this graph is not 
the same as the graph G7, that defines the test being analysed. In particular G 
is related to the word r being tested.) (2) We show that the existence of a large 
independent set in this graph G implies the proximity of r to a codeword of C™ 
(i.e., dgm(r)is small). (3) We show that this inconsistency graph is sparse if the 
T-soundness-error is small. (4) We show that the distance of C forces the graph 
to be special in that every edge is incident to at least one vertex whose degree 
is large. 


Definition of G. The vertices of G are indexed by pairs (b,i) with b € [rm] 
and i € [n]. Vertex (bı, i1) is adjacent to (bg, %2) if at least one of the following 
conditions hold: 


L. ÔCm-1(Tb] i) >T. 

2: ÔCm-1 (Tba i2) >T. 

3. by # bg and cbi and cb, ip are inconsistent, i.e., there exists some ele- 
ment j = (j1,---,Jm) € [n]™", with 7, = ii and fès, = tg such that 
Chia lj] Æ cozi lj], where j0 e [n]™—1 is the vector 7 with its beth 
coordinate deleted. 
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Independent Sets of G and Proximity ofr. It is clear that G has mn vertices. 
We claim next that if G has an independent set J of size at least m(n—d)+d+1 
then r has distance at most 1 — (|Z|/(mn))(1 — 7) to C”. 

Consider an independent set J = [,U---UJ, in G with J; of size ny being the 
set of vertices of the form (b,i),i € [n]. W.l.o.g. assume nı > --- > nm. Then, 
we have n1,n2 > n — d (or else even if nı = n and ng = n — d we'd only have 
Dno < n+ (m — 1)(n — d)). We consider the partial vector r’ € VA*r* xn 
defined as r'[i, j2,..-, fm] = C1,i[J2,---,Jm] for i € Å, and jo,...,jm € [n]. We 
show that r’ can be extended into a codeword of C™ and that the extended word 
is close to r and this will give the claim. 

First, we show that any extension of r’ is close to r: This is straightforward 
since on each coordinate i € Iį, we have r agrees with r’ on 1 — 7 fraction of 
the points. Furthermore J;/n is at least |I|/(mn) (since n; is the largest). So we 
have that r’ is at most 1 — ({J|/(mn))(1 — 7) far from r. 

Now we prove that r’ can be extended into a codeword of C™. Let Cy = Cz, 
be the projection (puncturing) of C to the coordinates in Jy. Let r” be the 
projection of r’ to the coordinates in J; x Ig x [n] x --- x [n]. We will argue below 
that r” is a codeword of C1 @ Cz @C™~?, by considering its projection to axis- 
parallel lines and claiming all such projections yield codewords of the appropriate 
code. Note first that the restriction of r’ to any line parallel to the 6-th axis is 
a codeword of C, for every b € {2,...,m}, since rj , is a codeword of C™—! for 
every? € J. Thus this continues to hold for r” (except that now the projection to 
a line parallel to the 2nd coordinate axis is a codeword of C2). Finally, consider 
a line parallel to the first axis, given by restricting the other coordinates to 
(i2,--.,4m), With ig € Ig. We claim that for every i1 € M, r”[i1,...,im] = 
C2.i2 {t1,---,4m]. This follows from the fact that the vertices (1,71) and (2,72) are 
not adjacent to each other and thus implying that c1, and c2,i, are consistent 
with each other. We conclude that the restriction of r” to every axis parallel 
line is a codeword of the appropriate code, and thus (by Proposition 2), r” is a 
codeword of C1 @C2@C™~*. Now applying Proposition 3 to the code Cy} @C™~! 
and its projection C1 ® Cz @ C™~* we get that there exists a unique extension 
of r” into a codeword c’ of the former. We claim this extension is exactly r’ 
since for every i € fy, ec, ;{3,k] = r'[i, j, k]. Finally applying Proposition 3 one 
more time, this time to the code C™ and its projection C1 & C™- t, we find that 
r’ = œ can be extended into a codeword of the former. This concludes the proof 
of this claim. 


Density of G. We now see that the small 7-soundness-error of the test translates 
into a small density y of edges in G. Below, we refer to pairs (b,2) with b € [m] 
and i € fn] as “planes” (since they refer to (m — 1)-dimensional planes in [n]™) 
and refer to elements of [n]™ as “points”. We say a point p = (pi1,..., Dm) lies on 
a plane (b, i) if pp = i. Now consider the following test: Pick two random planes 
(b1, i1) and (bg,%2) subject to the constraint bı 4 b2 and pick a random point p 
in the intersection of the two planes and verify that cy, i, is consistent with rfp]. 
Let x denote the rejection probability of this test. We bound « from both sides. 
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On the one hand we have that the rejection probability is at least the prob- 
ability that we pick two planes that are 7-robust and incident to each other 
in G (which is at least ==} — 2e) and the probability that we pick a point on 
the intersection at which the two plane codewords disagree (at least (d/n)™~?), 
times the probability that the codeword that disagrees with the point function 
is the first one (which is at least 1/2). Thus we get « > sin (a — 2e}. 

On the other hand we have that in order to reject it must be the case 
that either dgm~1(1s,,i,) > 7 (which happens with probability at most e} or 
dom-1(Tb,,i,) < T and p is such that rz, i, and c, disagree at p (which hap- 
pens with probability at most 7). Thus we have k < 7 + €. Putting the two 


m2 
together we have y < Z= (2e + Aor (T + e)). 


Structure of G. Next we note that every edge of G is incident to at least one 
high-degree vertex. Consider a pair of planes that are adjacent to each other in 
G. If either of the vertices is not T-robust, then it is adjacent to every vertex of 
G. So assume both are 7-robust. 

W.l.0.g., let these be the vertices (1,7) and (2,7). Thus the codewords c;,; and 
Co; disagree on the (m — 2)-dimensional surface with the first two coordinates 
restricted to i and 7 respectively. Now let S = {(k3,...,km) | e1,i[9, 3,---; km] £ 
c2 j (1, k3,...,km]} be the set of disagreeing tuples on this line. By the distance 
of C™-? we know |S| > d”~?, But now if we consider the vertex (b, ks) in G 
forb € {3,...,m} and k, such that there exists k1,...,km—g2 satisfying k = 
(k1,...,km-—2) E€ S, it must be adjacent at least one of (1,2) or (2,7) (it can’t 
agree with both at the point (i,j,k). Furthermore, there exists d such ky’s for 
every b € {3,...,m}. Thus the sum of the degrees of (1,7) and (2,7) is at least 
(m — 2)d, and so at least one has degree at least (m — 2)d/2. 


Putting It Together. From the last paragraph above, we have that the set of 
vertices of degree less than (m —2)d/2 form an independent set in the graph G. 
The fraction of vertices of degree at least (m — 2)d/2 is at most 2(ymn)/((m — 
2)d). Thus we get that if mn-(1 — 2(ymn)/((m — 2)d)) > m(n—d)+d+1, then 
ris 6-proximate to C™ for 6 < T + (1 — 7T): 2(ymn)/((m — 2)d). The lemma 
now follows by simplifying the expressions above, using the upper bound on y 
derived earlier. The calculations needed to derive the simple expressions can be 
found in the full version of this paper [12]. 


Next we improve the bound achieved on the proximity of 7 by looking at 
the structure of the graph G?, (the graph underlying the m-Product tester) 
and its “expansion”. Such improvements are a part of the standard toolkit in 
the analysis of low-degree tests based on axis parallel lines (see e.g., [7,6,16, 
17] etc.) We omit the proof of this part from this version — it may be found in 
the full version of this paper [12]. We state the resulting lemma and show how 
Theorem 1 follows from it. 
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Lemma 2. Let m be a positive integer and C be an |n,k,d|s code with the 
property d™—'/n™-1 > Z, If r e X” and c € C™ satisfy ô(r,c) < ł then 
d(r,c) < 8p(r). 


2m, 
Proof (Theorem 1). Let c = 214. (35) . We will prove that the m-Product 


Tester is c-robust for C™. Note that e < 2'® as required for the theorem, and 


yis < min{z - ( (1=)” T (H (as will be required below). 

The completeness (that codewords of C™ have expected robustness zero) 
follows from Proposition 2. For the soundness, consider any vector r € ©” and 
let p = p(r). If p > 1/c, then there there is nothing to prove since dgm(r) < 1 < 
c: p. So assume p < 1/e. 

Note that r has N ae at most ,/p. Furthermore, by the as- 


sumption on p, we have 3,/p < 3 t< 5 . (4=)” and so, by Lemma 1, we 


have Sam (e) <16. (3) i -2-,/p. Now using /p < 4/4 < 1i T we get 


mn 


dom(r) < ł z: Let v be a codeword of C™ closest to r. We now have (r, v) < t 


and a > ý, and so, by Lemma 2, we get ĝcm(r) = d(r,v) < 8p. This 
concludes the proof. 


4 Tanner Product Codes and Composition 


In this section we define the composition of two Tanner Product Codes, and 
show how they preserve robustness. We then use this composition to show how 
to test C™ using projections to C?. All proofs of Lemmas in this Section appear 
in the full version of the paper [12]. 

Recall that a Tanner Product Code is given by a pair (G, Csman). We start by 
defining a composition of graphs that corresponds to the composition of codes. 

Given an (N, M, D)-ordered graph G = {£),...,€.¢} and an additional 
(D,m,d)-ordered graph G’ = {@},...,€,,}, their Tanner Composition, denoted 
G@©G', is an (N, M -m,d)-ordered graph with adjacency lists {4} + |j € [M], 7’ € 


Iml}, where £7 = bjw, k 


G i 


Lemma 3 (Composition). Let Gı be an (N,M,D)-ordered graph, and Cı © 
EP be a linear code with C = TPC(G1, C1). Further, let Ga be an (D, m,d)- 
ordered graph and Cy C Xĉ be a linear code such that Cy = TPC(G3, C2). 
Then C = TPC(Gi@©@Go, C2) (giving a d-query local test for C). Furthermore if 
(Gi, Ci) 1S C1 -robust and (Go, C2) 1S Co-robust, then (Gy ©Go, C2) 1S C1 -c2 -robust. 


We continue by recasting the results of Section 3 in terms of robustness of 
associated Tanner Products. Recall that G}, denotes the graph that corresponds 
to the tests of C™ by the m-Product Tester, where C € ”. 

Note that G?, can be composed with G?,_, and so on. For m’ < m, define 
Gmm = Gm fm =m —1 anddefine GF, = CmOCm-1,m otherwise. Thus 
we have that C™ = TPC(GE m O" ): The following lemma (which follows 
easily from Theorem 1 and Lemma 3 gives the robustness of (Gi, Cr). 
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Lemma 4. Let C be an[n,k,d|s code with (d—1/n)* < $. Then (G}2,C?) is 
232 robust. 


Finally we define graphs HF so that CF = TPC(H?, C°). This is easily done 
t—2 
recursively by letting Hy = GZ, and letting HP = A ©H?,for t > 2. We 
now analyze the robustness of (H? , C°). 


Lemma 5. There exists a constant c such that the following holds: Let t be an 
integer and C be an [n,k,d]y code such thatd—1 > (1—ṣ;)}:n. Then (Hf, C?) 
is c’-robust. 


We are ready to prove Theorem 2. 


Proof (Theorem 2). Let c be the constant given by Lemma 5. Fix z and let 
C = C;, n = n; etc. (1.e., we suppress the subscript i below). Then C™ is an 
[N, K, D]q code, for N =n™, K = k™ and D = d™. Since d/n > 1 — 5+, we 
have C has relative distance d”/n™ > x. Furthermore, the rate of the code is 
inverse polynomial, i.e., N = n™ = (p(k))™ < poly(k™) = poly(K). Finally, we 
have C™ = TPC(H? C*), where ( C?) is a c'°82™-robust tester for 


n 
loga m? loga m? 
C™ and this tester has query complexity O(n). From Proposition 1 we get that 
there is a tester for C thatmakes O(n2c0(°82™)) = poly log N queries. 


Acknowledgments 


We wish to thank Irit Dinur, Oded Goldreich and Prahladh Harsha for valuable 
discussions. 


References 


l. Noga Alon, Tali Kaufman, Michael Krivelevich, Simon Litsyn, and Dana Ron. 
Testing Reed-Muller codes. In Proc. RANDOM 2003, pages 188-199, 2003. 

2. Sanjeev Arora. Probabilistic checking of proofs and the hardness of approximation 
problems. PhD thesis, University of California at Berkeley, 1994. 

3. Sanjeev Arora, Carsten Lund, Rajeev Motwani, Madhu Sudan, and Mario Szegedy. 
Proof verification and the hardness of approximation problems. Journal of the 
ACM, 45(3):501-555, May 1998. 

4. Sanjeev Arora and Shmuel Safra. Probabilistic checking of proofs: A new charac- 
terization of NP. Journal of the ACM, 45(1):70-122, January 1998. 

5. Sanjeev Arora and Madhu Sudan. Improved low-degree testing and its applications. 
In Proc. STOC97, pages 485-495, El Paso, Texas, 4-6 May 1997. 

6. Laszl6 Babai, Lance Fortnow, Leonid A. Levin, and Mario Szegedy. Checking 
computations in polylogarithmic time. In Proc. STOCQ9I, pages 21-32. 1991. 

7. László Babai, Lance Fortnow, and Carsten Lund. Non-deterministic exponential 
time has two-prover interactive protocols. Computational Complexity, 1(1):3—40, 
1991. 

8. Mihir Bellare, Don Coppersmith, Johan Hastad, Marcos Kiwi, and Madhu Su- 
dan. Linearity testing over characteristic two. IEEE Transactions on Information 
Theory, 42(6):1781-1795, November 1996. 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


26. 


Robust Locally Testable Codes and Products of Codes 297 


. Mihir Bellare, Shafi Goldwasser, Carsten Lund, and Alex Russell. Efficient proba- 


bilistically checkable proofs and applications to approximation. In Proc. STOC93, 
pages 294-304. ACM, New York, 1993. 

Mihir Bellare and Madhu Sudan. Improved non-approximability results. In Proc. 
STOC94, pages 184-193, Montreal, Quebec, Canada, 23-25 May 1994. 

Eli Ben-Sasson, Oded Goldreich, Prahladh Harsha, Madhu Sudan, and Salil Vad- 
han. Robust PCPs of proximity, shorter PCPs and applications to coding. In Proc. 
STOCO04, (to appear), 2004. 

Eli Ben-Sasson and Madhu Sudan. Robust Locally Testable Codes and Prod- 
ucts of Codes. Available at Electronic Colloquium on Computational Complexity 
http: //eccc.uni-trier.de/eccc-reports/2004/TRO4-046/index.htm1 

Eli Ben-Sasson, Madhu Sudan, Salil Vadhan, and Avi Wigderson. Randomness 
efficient low-degree tests and short PCPs via e-biased sets. In Proc. STOCO3, 
pages 612—621, 2003. 

Manuel Blum, Michael Luby, and Ronitt Rubinfeld. Self-testing/correcting with 
applications to numerical problems. Journal of Computer and System Sciences, 
47(3):549-595, 1993. 

Irit Dinur and Omer Reingold. Assignment-Testers: Towards a Combinatorial 
Proof of the PCP-Theorem. Manuscript, 2004. 

Uriel Feige, Shafi Goldwasser, Laszlo Lovasz, Shmuel Safra, and Mario Szegedy. 
Interactive proofs and the hardness of approximating cliques. Journal of the ACM, 
43(2):268—292, 1996. 

Katalin Friedl, Zsolt Hatsagi, and Alexander Shen. Low-degree tests. In Proc. 
SODA94, pages 57—64, 1994. 

Katalin Friedl and Madhu Sudan. Some improvements to total degree tests. In 
Proc. Israel STCS, pages 190-198, Tel Aviv, Israel, 4-6 January 1995. 

Oded Goldreich and Muli Safra. A Combinatorial Consistency Lemma with ap- 
plication to the PCP Theorem. In SJAM Jour. on Comp., Volume 29, Number 4, 
pages 1132-1154, 1999. 

Oded Goldreich and Madhu Sudan. Locally testable codes and PCPs of almost- 
linear length. In Proc. FOCS02, Vancouver, Canada, 16-19 November 2002. 

F. J. MacWilliams and Neil J. A. Sloane. The Theory of Error-Correcting Codes. 
Elsevier/North-Holland, Amsterdam, 1981. 

Alexander Polishchuk and Daniel A. Spielman. Nearly linear-size holographic 
proofs. In Proc. STOC94, pages 194-203, Montreal, Canada, 1994. 

Ran Raz and Shmuel Safra. A sub-constant error-probability low-degree test, and 
a sub-constant error-probability PCP characterization of NP. In Proc. STOC97, 
pages 475-484. ACM Press, 1997. 

Ronitt Rubinfeld and Madhu Sudan. Robust characterizations of polynomials with 
applications to program testing. SIAM J. Comp., 25(2):252—271, 1996. 

Madhu Sudan. Algorithmic introduction to coding theory. Lecture notes, Available 
from http://theory.csail.mit.edu/madhu/FTO1/, 2001. 

R. Michael Tanner. A recursive approach to low complexity codes. JEEE Trans- 
actions of Information Theory, 27(5):533—547, September 1981. 


A Stateful Implementation 
of a Random Function 
Supporting Parity Queries over Hypercubes 


Andrej Bogdanov and Hoeteck Wee 


Computer Science Division 
University of California, Berkeley 
{adib, hoeteck }@cs .berkeley.edu 


Abstract. Motivated by an open problem recently suggested by Gol- 
dreich et al., we study truthful implementations of a random binary 
function supporting compound XOR queries over sub-cubes of the hy- 
percube {0,1}”. We introduce a relaxed model of an implementation, 
which we call a stateful implementation, and show how to implement 
the desired specification in this model. The main technical construction 
is an algorithm for detecting linear dependencies between n dimensional 





hypercubes, viewed as characteristic vectors in ploy Using coding the- 
oretic techniques, we first exhibit a randomized algorithm for detecting 
such dependencies. We then show how a recent approach by Raz and 
Shpilka for polynomial identity testing in non-commutative models of 
computation can be applied to obtain a deterministic algorithm. 





1 Introduction 


In arecent paper, Goldreich, Goldwasser and Nussboim [3] initiated the study of 
efficient pseudorandom implementations of huge random objects — objects so big 
that they cannot be represented using bounded resources, such as randomness or 
time (in particular, these objects have size that is exponential in the running time 
of the applications), but for which we can obtain approximations good enough 
for many algorithmic and cryptographic applications. A celebrated example from 
cryptography is the construction of a pseudo-random function from any one way 
function [2]: Even though a truly random boolean function on n input bits cannot 
be specified by fewer than 2” random bits, in many cryptographic applications 
this infeasible object can be approximated by a pseudo-random function that can 
be specified using only poly(n) random bits and evaluated on arbitrary inputs 
in poly(n) time. 


1.1 Stateful and Stateless Implementations 


Since the work of Goldreich et al. is somewhat motivated by cryptography, they 
consider only “stateless” implementations of huge random objects. An imple- 
mentation refers to a polynomial-time (oracle) machine that computes the huge 
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object; in the case of a random boolean function, the machine takes as input a 
string of n bits and outputs the value of the function at that point. In a stateless 
implementation J, the answer to a query posed to J cannot depend on queries 
previously seen by / and their answers. This property of the implementation is 
often important in cryptographic settings, where multiple parties share the same 
copy of the huge object in question. For example, a common technique in cryp- 
tography is to design protocols in the random oracle model, where each party 
has access to the same infinite sequence of random bits. To obtain an implemen- 
tation of the protocol, one replaces the random oracle with a pseudo-random 
function. As we want the parties in the protocol to share the same pseudo- 
random function, it is important that the implementation of the random oracle 
by a pseudo-random function be independent of the queries seen by a particular 
party in the protocol; namely, the implementation must be stateless. 

In addition to cryptography, Goldreich et al. also consider algorithmic ap- 
plications of huge random objects. For example, they imagine a scenario where 
one wants to run experiments on, say, random codes. They observe that global 
properties of these codes, such as having large minimum distance, may not be 
preserved when the randomness of the code is replaced by a pseudo-random gen- 
erator. This leads to the problem of implementing huge random objects that are 
guaranteed to preserve a certain property, such as codes with good minimum 
distance. 

Unlike in the cryptographic setting, it is not clear that a stateless imple- 
mentation (the only type allowed by the model of Goldrech et al.) gives any 
advantage over a “stateful” one. In other words, it may be possible to do more 
by generating the desired huge random object “on the fly” rather than subscrib- 
ing to an implementation predetermined by the random tape of our machine. In 
particular, it would be interesting to know whether there exists a natural specifi- 
cation that allows a stateful implementation but not a stateless one. We suspect 
that the specification considered in this paper, suggested for study by Goldreich 
et al. — a random boolean function supporting XOR queries over hypercubes — 
may provide a separation between stateful and stateless perfect implementations 
in the random oracle model. 


1.2 Random Functions Supporting Complex Queries 


Goldreich et al. observe that, assuming the existence of one-way functions, if 
a specification can be close-implemented'’ in the random oracle model, then it 
can also be implemented by an ordinary probabilistic polynomial-time machine. 
Moreover, this transformation preserves truthfulness’: Namely, to obtain a truth- 


' In fact, it is sufficient that the specification be pseudo-implementable. Note that the 
terms close-implementable, pseudo-implementable and truthful are technical terms 
defined in [3]. 

* Intuitively, truthfulness requires that an implementation of Type T objects gen- 
erates only objects of Type T. In particular, a random function is not a truthful 
implementation of a random permutation even though they are indistinguishable to 
a computationally bounded adversary. 
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ful pseudo-implementation of a huge random object, it is sufficient to construct 
such an implementation in the random oracle model. This is a common technique 
in cryptography, used among other things in the construction of pseudo-random 
permutations [4]. Though the transformation is only shown to hold for stateless 
implementations, we observe that it also works for stateful ones (this is because 
the definition of pseudo-randomness in the context of pseudo-random functions 
allows from stateful adversaries). 

In particular, this observation implies that random functions have a trivial 
truthful pseudo-implementation. However, one may ask whether it is possible 
to truthfully implement random functions supporting queries beyond evalua- 
tion on arbitrary inputs. One variant proposed by Goldreich et al. asks for the 
implementation of a random function f : {0,1}" — {0,1}, augmented with 
queries regarding the XOR of the values of f on arbitrary intervals of {0,1}" 
(with respect to the lexicographic ordering of n bit strings.) Note that a trivial 
implementation of f in the random oracle model cannot hope to answer such 
queries efficiently, as they may involve XORing exponentially many bits. How- 
ever, Goldreich et al. show how, using a suitable data structure, one can obtain 
a stateless perfect implementation of f that answers queries in O(n?) time. As 
perfect implementations are always truthful, this construction yields a truthful 
pseudo-implementation by an ordinary machine. 

We observe that a simpler construction for implementing random functions 
supporting interval-XOR queries with O(n) running time can be achieved as 
follows:let f’ : {0,1}” — {0,1} be the random oracle, and return f’(a—1)@f'(£) 
as the answer to the query (a, 8), corresponding to the value @,<,<g f(z). Here, 
a—1 denotes the n-bit binary string that immediately precedes a in lexicographic 
order, and we specify f’(0” — 1) = 0. The underlying idea is a simple change of 
basis: instead of specifying a random function f by its values at all x € {0,1}",we 
specify the value of f using the values Die: f(y). Note that this implementation 
makes only 2 queries into the random oracle per interval-XOR query, which is 
optimal (in an amortized sense). Unfortunately, this construction, unlike that 
by Goldreich et al., does not yield a truthful close-implementation of random 
functions supporting any symmetric interval query. 

As a follow-up to their work on interval queries, Goldreich et al. propose the 
following more general question: 


Open Problem. [3] Provide a truthful close-implementation in the random 
oracle model of the following specification. The specification machine defines 
a random function f : {0,1}" — {0,1}, and answers queries that succinctly 
describe a set S, taken from a specific class of sets, with the value „es f(z). 
A natural case is the class of sub-cubes of {0,1}; that is, a set S is specified by 
a pattern o in {0,1,*}” such that S is the set of points in {0,1}” that match 
the pattern ø. 


We suspect that the technique of Goldreich et al. for implementing a random 
function supporting interval queries does not extend to the case of hypercube 
queries, though we have not been able to show a negative result confirming our 
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intuition. Instead, we show how to obtain a stateful perfect-implementation of 
this specification in the random oracle model. 


1.3 Main Contributions 
Our main contributions are the following: 


1. We propose a notion of stateful implementations of huge random objects and 
reduce the problem of constructing a stateful implementation of a random 
binary function supporting compound XOR queries over sub-cubes of the 
hypercube {0,1}"to an algorithmic problem of hypercube linearity testing. 


2. We then present two algorithms for hypercube linearity testing: a random- 
ized algorithm using coding theoretic techniques, and a deterministic based 
on non-commutative polynomial identity testing. It follows from the first al- 
gorithm that there is a stateful close implementation of the afore-mentioned 
specification in the random oracle model, and from the second, a stateful 
perfect implementation. 


In fact, the second algorithm subsumes the first, but we still include the latter 
as we feel that the technique used is fairly insightful and the analysis can in fact 
be used to construct quantum states of provably superpolynomial tree size. 


2 Preliminaries 


Let a € {0,1,*}". The hypercube’ H(a) is the set of alla € {0,1}” that match 
the string a, namely such that 


0, if ali] = 0 
m ee ae if ali] =1 
Oor 1, if alt] =. 


As in Goldreich et al., we specify the huge random object in question by a 
computationally unbounded probabilistic Turing machine that halts with prob- 
ability one. The huge random object is determined by the input-output relation 
of this machine when the random tape is selected uniformly at random from 
E 


A Random Function Supporting XOR Queries on Hypercubes 
INPUT A query a € {0,1, *}” 

RANDOM TAPE A sequence of functions fi, fo,..., where fx : {0,1}* — {0,1} 
OUTPUT The value Bye q (a) flaj(2) 


We are interested in efficient implementations of this specification, namely 
ones that can be obtained by Turing machines that run in time polynomial in 
the length of the input. 





? In [3] and in the introduction, we use the term sub-cubes of the hypercube {0, 1}”. 
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2.1 Stateful Implementations 


Following Goldreich et al., we say machine / 1s a (stateful) implementation spec- 
ification S with respect to a machine M of that makes queries to a protocol 
party, if (1) On the qth query x, J runs in time polynomial in g and zx, and (2) 
The distribution Dg of the transcript M(1") = S is indistinguishable from the 
distribution Dr of M(1”) + I. Specifically: 


1. If Ds and Dy; are identical for every M, we say that I perfectly implements 


S; 

2. If Ds and Dy have negligible (n”) statistical difference for all M that 
make poly(n) queries, we say that I closely implements S; 

3. If Ds and Dy are indistinguishable for all M that run in polynomial time, 
we say that J pseudo-implements S. 


An implementation J is truthful with respect to S if for every sequence of 
queries £1,..., £q, the support of the distribution [(2;),...,£(aq) is contained 
in the support of S(z1),...,S(zq_); namely, if the implementation never provides 
answers that are inconsistent with the specification. Note that perfect implemen- 
tations are always truthful. An oracle Turing machine J’ is an implementation 
of S in the random oracle model if the distribution of M(1") © S is indistin- 
guishable from the distribution of M(1") — I* over a random oracle R. As for 
stateless implementations (see Theorem 2.9 of [3]), we have the following: 


Proposition 1. Suppose that one-way functions exist. Then any specification 
that has a pseudo-implementation in the random oracle model also has a pseudo- 
implementation by an ordinary machine. Moreover, if the former implementation 
is truthful then so is the latter. 


Since perfect implementations are always truthful, for our purposes it will be 
sufficient to provide a perfect implementation of a random function supporting 
queries on hypercubes in the random oracle model. The heart of this implemen- 
tation consists of an algorithm for the problem of hypercube linearity testing, or 
HYPERCUBE-LIN, which we describe next. 


2.2 Reduction to Hypercube Linearity Testing 


We call collection ofhypercubes Ai,..., Hm C {0,1}” linearly dependent if there 
exists a non-empty set S C {1,2,...,m} such that for all points x € {0,1}", z 
is contained in an even number of hypercubes amongst the subset of hypercubes 
{H; | i € S}. For any such set S, we write jies Hi = 0. Equivalently, we 
may view each hypercube H; as a vector in Bo where Fə = {0,1} is the 
two element field and H;[z] = 1 if x € H;, and 0 otherwise. In this notation, 
linear independence between hypercubes translates into linear independence of 
the corresponding vectors. 


HYPERCUBE-LIN: Given q strings a1,...,@q € {0,1,*}”, accept iff the hyper- 
cubes H(a,),...,H(a,) are linearly dependent. 
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Note that standard techniques for testing linear independence, such as Gaus- 
sian elimination, do not apply directly for this problem because we are dealing 
with vectors whose length is exponential in the size of the input. However, we 
will still be able to show the following: 


Theorem 1. There is a deterministic polynomial-time algorithm for the prob- 
lem HYPERCUBE-LIN. 


In fact, we will begin with a randomized algorithm for HYPERCUBE-LIN, 
which nicely illustrates the coding theoretic nature of the hypercube linearity 
testing problem. We then argue that the test performed by the algorithm can 
be viewed, in some sense, as an application of polynomial identity testing. Even 
though we don’t know, in general, how to derandomize polynomial identity test- 
ing, in our case the derandomization can be performed using a recent identity 
testing algorithm for non-commutative formulas of Raz and Shpilka [5]. 


Theorem 2. There exists a stateful perfect implementation of a random func- 
tion supporting XOR queries on hypercubes in the random oracle model. 


Proof. Let A be the algorithm from Theorem 1, and R be the random oracle. 
First, we consider the following (promise) search problem: 


INPUT Strings a1,...,@q E {0,1,*}” and b € {0,1,*}", such that the hypercubes 
H(a,),...,H(aq) are linearly independent 

PROBLEM If H(b) is linearly independent from H(aj),...,H (ag), output Ø. 
Otherwise, output coefficients cy,...,¢q E F2 such that H(b) = $}; ciH (a;). 


It is not difficult to see that, by a self reduction argument, we can obtain a 
polynomial time algorithm A’ for this problem using black box access to A (in 
fact, 2g invocations of A suffices). With this in hand, we implement a random 
function supporting XOR queries on hypercubes as follows: After seeing queries 
@4,.--,@q—1, the implementation keeps track of a subset of queries {ax : k € B}, 
where B C fq — 1] is chosen such that the subset of hypercubes {H(a;):k € B} 


form a basis for the set of vectors {H (ap) € pou : k € |q — 1}}. On query ag, 
we run the algorithm A’ on inputs {ax : k € B} and aq. If the algorithm returns 
Ø, then we return R(|B| + 1), which is a fresh random bit, and add a, to the 
set B. Otherwise, the algorithm A’ outputs coefficients c1,...,Cq—1 € Fe, and 
we return the value pie Rit). 

We show this is a perfect implementation, by induction on g. Let us assume 
that the specification transcript and implementation transcript are statistically 
indistinguishable after q — 1 queries. At query q, there are two possibilities: If 
H (aq) is linearly dependent in H(a;),...,H(@g—1), then both in the implemen- 
tation and in the specification the answer to the gth query is determined by the 
previous answers, so by the inductive hypothesis the new transcripts are indis- 
tinguishable. If H(a,) is linearly independent in H(aj),...,H(a@g_-1), then the 
answer to the gth query in the specification is statistically independent from all 
previous answers. By construction, this is also true in the implementation, so 
again, the new transcripts are indistinguishable. 
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3 Algorithms for Hypercube Linearity Testing 


In this section, we present two algorithms for HYPERCUBE-LIN, thereby com- 
pleting our implementation of a random function supporting XOR queries on 
hypercubes. 


3.1 A Randomized Algorithm for HYPERCUBE-LIN 


Let F = Fos be a field of characteristic 2 of size 2°, where s = [log 3n]. For each 


a € {0,1,*}", we define a polynomial pg = 2122--+- Zn over FE[z1, y1,- -., Zn, Yn}, 
where: 
Li if Qi = 0 
Zi = Vi asl 


Ti +y; ifa;=* 


Note that H(a) is a homogeneous polynomial of total degree exactly n, and 
that pa = È sc H(a) Pz- 


Lemma 1. Let F be any field of characteristic two. Then H (a1) +--+ H (am) = 
0 if and only ifpa, +-+ Pa is the zero polynomial in F{z1, y1,..-, En, Yn]: 


Proof. Each point x in {0,1}” is represented by a unique monomial ps from 
F[z1, Y1, .-- Zn, Yn]. Therefore, each point x in {0,1}” appears an even number 
of times in H(a,),..., (am) iff each of the corresponding monomials pẹ has an 
even integer coefficient in pg, +---+Da,,- In addition, since F has characteristic 
2, the latter condition is equivalent to pa, +:::+DPa,, being identically zero. 


We may now define a binary encoding CH(a) of hypercubes H(a), which is 
obtained by concatenating the Reed-Muller code associated with the polynomial 
Pa With the Hadamard code. More precisely, given any a € {0,1,*}, we define 
Cra) : {0,1} @+)* — {0,1} as follows: 


Cua) (1,91; cee Ean T) = (Pa(21, Y1, a ‘Tms Ym), T) 


using the first 2ns bits of the input to CH(a) to pick £1,Y1,.--, En, Yn € F, and 
the remaining s bits to pick 7 € {0,1}°. 


Lemma 2. {CH(a) | a € {0,1, *}"} is a binary encoding ofhypercubes in {0, 1}” 
with the following properties: 


I, (large distance) It has relative distance 1/3. 

2. (locally encodable) There is a O(nslog s) algorithm that computes C'y (a) (r) 
on input a and r € {0,1} @r+)s, 

3. (linearity) For all a,a' € {0,1,*}”, CH(a)+H(o') = CH(a) + Cua’): 


Proof. By the Schwartz-Zippel Lemma [6,7], for any a # a’ € {0,1,*}”, Pa 
and pa; evaluate to different values on at least a 2/3 fraction of values in F?”. 
Upon concatenating with the Hadamard code, the minimum relative distance 
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becomes 1/3. Next, observe that we can write down pa and evaluate pg at any 
input in time O(nslogs). It follows that we also compute Cy,q)(r) in time 
O(ns log s). Finally, linearity follows from the fact that both the encoding as 
polynomials and the Hadamard code are linear. 


Randomized Algorithm for HYPERCUBE-LIN 


1. Fix £ = O(q). Choose rj,...,7¢ € {0,1}@"+)* uniformly at random. 
2. Construct the q x £ matrix M over F2, where Mij = CH(a;) (rj), that is: 


Cy(a1)(71) Cua) (r2) --- CHa) (Te) 
CH(az)(71) CH(az)(T2) re Cy(a2) (Te) 


Cu(a,)(T1) CHya,)(T2) --- Cra, (re) 


3. Compute the rank of M over Fp. Accept if the rank is less than q; reject 
otherwise. 


Proposition 2. There is a coRP-algorithm for HYPERCUBE-LIN running in 
time O(@ + q?ns log s). 


Proof. Fix any S C [|q]. It follows from linearity and Lemma 1 that 


`. H(a;)=0_ if and only if `S CHa,(r) =0 Yre {0, Jeanty (1) 
iES iES 


Therefore, if H (a1), ..., H (aq) are linearly dependent, there is some S for which 
(1) holds. Then, the rows of M identified by S add up to 0, and thus M has rank 
less than q. On the other hand, if H (aı),..., H(aq) are not linearly dependent, 
then for all S C [q], } ics CH(a:) 18 not the zero codeword", and thus 


Er DA Cua:)(73) =0 Yj= Eins < (=) 


Tipes he 


Now, the probability that M has rank less than q is equal to the probability that 
there is some S C |g] for which (1) holds, which by a union bound is at most 
29.(2/3)* < 1/2. Finally, computing M takes time O(géns log s), and performing 
Gaussian elimination on M takes time O(q?£), which yields a total running time 
of O(g? + q?ns logs). 


* In fact, ie s CH a;) is not necessarily a codeword in the code defined in Lemma 2, 
but it is still a codeword in the code obtained by concatenating a Reed Muller code 
with a Hadamard code. Hence, the analysis for the relative distance of the code also 
shows that } ies CH (a,) has relative distance at least 1/3 from the zero codeword. 
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3.2 A Deterministic Algorithm for HYPERCUBE-LIN 


The randomized algorithm for hypercube linearity testing is based on Lemma 1, 
which allows us to reduce testing a particular dependency between hypercubes 
to a polynomial identity test. In the actual algorithm, the power of randomness 
is used twice. First, randomness allows us to efficiently perform the polynomial 
identity test from Lemma 1. Second, using standard amplification of success 
probabilities we can take a union bound over the exponentially many possible 
linear dependencies between hypercubes in the correctness proof for the random- 
ized algorithm. 

Even though polynomial identity testing remains notoriously hard to deran- 
domize, Raz and Shpilka [5] recently found a deterministic algorithm that works 
for certain alternate and restricted models of computation. In particular, their 
algorithm works for formulas over arbitrary fields in non-commuting variables 
(i.e., formulas in the noncommutative ring F{z1,..., £n}), and for YHE circuits 
— depth three multilinear arithmetic circuits with a plus gate at the root. 

The formal polynomial pı +. ..+pm in Lemma 1 can be trivially computed by 
an XIS circuit. This immediately gives a deterministic way of testing whether 
a particular linear relation Hı +...+Hm = 0 is satisfied over F2. However, we are 
interested in testing whether any one of the exponentially many relations cı Hı + 
..-+CmH, = 0 holds, or equivalently, if there exists coefficients c1, ..., €m E F2, 
not all zero, such that c1p;+...+¢CmPm = 0 as a polynomial in Fo[z;, y;]. We will 
show that a construction, along the lines of Raz and Shpilka’s algorithm, works 
for this problem as well. Instead of trying to come up with a general theorem for 
testing linear dependencies between polynomials, we will focus on polynomials 
representing hypercubes, though it may be possible to extend the analysis to a 
somewhat more general scenario. 

Let F now be an arbitrary finite field, and p1, . . . , Pm be polynomials in F[z;,], 
where 7 € [n],7 € [m]and each pp, k € [m]has the following form: 


pr(zij) = [] > oity, (2) 


i=1 j=l 


where ak, € Fare constants. It is easy to check that the polynomials in Lemma 1 
are written in this form. We are interested in determining, in time polynomial 
in m and n, whether there exist coefficients ¢ĉ1,...,Cm E F such that cipi +...+ 
CmPm is the zero polynomial over F. Since this is a multilinear polynomial, this 
is equivalent to asking whether p1,...,Pm are linearly dependent as elements 
of the vector space V over F generated by the monomials 2,;, ...2%nj,,, where 
d1,++-;jn Tange over [m]. 

As in [5], the analysis works by induction on n. When n = 1, we have 
Pk(L1j) = Do O7;%1j, SO the vectors pp € V are independent if and only if 
the matrix M € F™*™ with M[j, k] = af; has full rank. Using Gaussian elimi- 
nation this can be checked in, say, O(m? log |F|) time. 

We now show how to reduce a problem of degree n to one of degree n — 1. 
The trick is to look at the following partial expansion of the polynomials px: 
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m 


k 
Pr(£ij) = ` OF 5, XZja Tli T2ja ` qk (X3. aa): 
ji j2=i 


where q; 1S given by 


n m 
qk (X(3...n)j) = IkhT3j, T4js Laj) = I] S af riz. 


i=3 j=1 


Now consider a linear combination P(z;;) = $O p-] cepe(zi;). We can expand 
this as 


P(x) = 3 L151 T2jz let aes: Chk ((3...n)3): 


jJisj2=1 k=1 


From this expansion, we see that P = 0 in F[z;,] if and only if for all j1, j2 € [rn]: 


N afn 054, ' Ckak(T(3...n)5) = 0 in F[x(3...n);]- (3) 
k=1 


Consider the matrix M € pr xm, whose rows are indexed by pairs (j1, j2) € 
[m] x [m] such that M[(j1, j2), k] = af; @5;,- Choose a subset S of m rows such 
that the rows indexed by S$ span the row space of M. Then the constraints (3) 
are all satisfied if and only if 


For alls € S, $ MIs, k] - ceqe(2(3...n)j) = 0 in Ffea...n)z]. (4) 
k=l 


We now want to rewrite the set of constraints (4) as a single constraint. For this, 
we introduce m additional formal variables y, with s € S. Constraints (4) are 
satisfied 1f and only 1f 


Noys Ss M |s, k] - cege(%(3...n)j) = 0 in Flys, 2(3...n)5]. (5) 
ses k=1 


Finally, let 
Tk(Ys, T(3.. A) TON es M|\s, kys) qk(T(3.. n)j)s 


and note that constraint (5) is satisfied if and only if J CKT k (Ys, E(3..n)j) = 0 
in Flys, £(3.,.n);]- TO summarize: 


Lemma 3. The polynomials pı,..., Pm are linearly independent in F|x;;| if and 
only if the polynomials r1,...,m are linearly independent in Flys, £(3...n);] ° 


> Moreover, the p; satisfy a particular dependency X` cipi = 0 if and only if the ri 
satisfy the same dependency. This can be used to find linear dependencies between 
hypercubes, which makes the decision to search reduction in the proof of Proposi- 
tion 1 unnecessary. 
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Now we can apply the induction hypothesis to the polynomials r, which have 
the prescribed form (2). The bottleneck in the running time of the algorithm is 
computing the linearly independent set S, which can be done in O(m?* log |F]) 
time by Gaussian elimination. This yields a total running time of O(m‘n log |F}), 
and concludes the proof of Theorem 1. 


4 Extensions 


4.1 A Connection with Quantum Computation 


The following problem was brought to our attention by Aaronson in his recent 
work addressing skepticism of quantum computing [1]. The construction below 
implies the existence of explicit (constructible in deterministic polynomial-time) 
quantum states of provably superpolynomial tree size. The argument is similar 
to the proof of Proposition 2. 


Proposition 3. For every 6 < 1/4, there is a polynomial-time constructible 
k x n binary matrix M with k = n°, such that a random k x k submatrix of M 
has rank at least k — 1 with constant probability. 


Aaronson’s original argument requires M to have full rank with constant 
probability, but it is not difficult to see that his argument applies even if M has 
only rank k — 1 with constant probability. 


Proof. Fix € > 0 and choose ko such that k = ko log k2*£. Consider the concate- 
nation of a Reed-Solomon code with codeword size kg** (over smallest possible 
alphabet size) with a Hadamard code. The resulting binary code C has length 
at most n = ktt% and relative distance 1/2 — 7, where 7 = 1/k1t+*-°™), Pick a 
basis €1,...,e% of {0,1}* and take M to be the k x n matrix whose ith row is 
the codeword C'(e;). 

Now consider choosing a random k x k submatrix of M. Since k = O(n'/4), 
with probability 1 — o(1), this is equivalent to choosing an independent set of k 
columns Mı,..., Mp of M with repetition. For any nonzero vector x € {0,1}*, 
the probability that all dot products x?Mj,...,27M, vanish is at most (1/2 + 
n)*. It follows that the expected number of nonzero x for which all these dot 
products vanish is at most 2% . (1/2-+7)* < exp(2k~*) = 1+ (1). By Markov’s 
inequality, the number of such z is less than two with probability at least, say, 
1/3. If this is the case, then the columns of M satisfy at most one linear relation, 
so that the rank of M is at most k — 1. 


4.2 Towards a Stateless Implementation? 


We discuss our conjecture regarding a possible separation between stateful and 
stateless perfect implementations in the random oracle model. We say that a set 
S of vectors in F3” admits an efficient basis B where B is a (standard) basis for 
F2” if every vector v € S can be written as the sum of poly(n) vectors in B and 
the basis representation can be computed in poly(n) time. 
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Proposition 4. If S admits an efficient basis, then there is a perfect (stateless) 
implementation of a random function f : {0,1}" — {0,1}, and answers queries 
that succinctly describe a set S, taken from a specific class of sets, with the value 


Dres f(z). 


It is easy to see that intervals of {0,1}” admit an efficient basis, namely the 
collection of vectors {v; € F2” ,i = 1,2,...,2”} where v; is the vector whose first 
į positions are ones and whose remaining positions are zeros. This observation 
underlies our construction for implementing random functions support interval- 
XOR queries in Section 1.2. On the other hand, we do not know if the hypercubes 
of {0,1}”" admit an efficient basis. 


Acknowledgements 


We thank Luca Trevisan and the anonymous referees for comments on an earlier 
version of this paper. 


References 


1. Scott Aaronson. Multilinear formulas and skepticism of quantum computing. In 
Proceedings of the 36th ACM Symposium on Theory of Computing, 2004. 

2. Oded Goldreich, Shafi Goldwasser, and Silvio Micali. How to construct random 
functions. Journal of the ACM, 33(4):210—217, 1986. 

3. Oded Goldreich, Shafi Goldwasser, and Asaf Nussboim. On the implementation of 
huge random objects. In Proceedings of the 44th IEEE Symposium on Foundations 
of Computer Science, pages 68—79, 2003. Preliminary full version at 
http: //www.wisdom.weizmann.ac.il/“oded/p toro.html. 

4. Michael Luby and Charles Rackoff. How to construct pseudorandom permutations 
from pseudoranom functions. SICOMP, 17:373-386, 1988. 

5. Ran Raz and Amir Shpilka. Deterministic polynomial identity testing in non commu- 
tative models. In Proceedings of the 17th Conference on Computational Complexity, 
2004. To appear. 

6. J. T. Schwartz. Fast probabilistic algorithms for verification of polynomial identities. 
Journal of the ACM, 27(4):701-717, 1980. 

7. R. E. Zippel. Probabilistic algorithms for sparse polynomials. In Proceedings of 
EUROSAM 79, pages 216-226, 1979. 


Strong Refutation Heuristics for Random k-SAT 


Amin Coja-Oghlan', Andreas Goerdt’, and André Lanka? 


' Humboldt-Universitit zu Berlin, Institut fiir Informatik 
Unter den Linden 6, 10099 Berlin, Germany 


coja@informatik.hu-berlin.de 
* Technische Universität Chemnitz, Fakultät fiir Informatik 
StraBe der Nationen 62, 09107 Chemnitz, Germany 


{goerdt, lanka}@informatik.tu-chemnitz.de 


Abstract. A simple first moment argument shows that in a randomly 
chosen k-SAT formula with m clauses over n boolean variables, the frac- 
tion of satisfiable clauses is at most 1 — 27% + o(1) as m/n — oo almost 
surely. In this paper, we deal with the corresponding algorithmic strong 
refutation problem: given a random k-SAT formula, can we find a cer- 
tificate that the fraction of satisfiable clauses is at most 1 — 2~* + o(1) 
in polynomial time? We present heuristics based on spectral techniques 
that in the case k = 3, m > In(n)°n°/? and in the case k = 4, m > Cn? 
find such certificates almost surely, where C denotes a constant. Our 
methods also apply to some hypergraph problems. 


1 Introduction and Results 


The k-SAT problem — given a set of &-clauses, i.e. disjunctions of k literals 
over a set of boolean variables, decide whether there exists an assignment of 
the variables that satisfies all clauses — is the generic NP-complete problem. In 
addition to the decision version, the optimization version MAX k-SAT — given a 
set of k-clauses, find an assignment that satisfies the maximum number of clauses 
—is of fundamental interest as well. However, Hastad [18] has shown that there 
is no polynomial time algorithm that approximates MAX k-SAT within a factor 
better than 1—2-*, unless P = MP. Hence, it is NP-hard to distinguish between 
instances of kK-SAT in which a (1 —¢)-fraction of the clauses can be satisfied, and 
instances in which every truth assignment satisfies at most a (1 —2~* +¢)-share 
of the clauses for any £ > 0. Indeed, Hastad’s NP-hardness result is best possible, 
as by picking a random assignment, we can satisfy a (1 — 2~*)-fraction of the 
clauses in polynomial time. 

These hardness results motivate the study of heuristics for k-SAT or MAX 
k-SAT that are successful at least on a large class of instances. From this point 
of view, the satisfiability problem is interesting in two respects. First, one could 
ask for heuristics for finding a satisfying assignment (in the case of k-SAT) or a 
“good” assignment (in the case of MAX k-SAT). This problem has been studied 
e.g. by Flaxman [11], who has shown that in a rather general model of random 
satisfiable formulas a satisfying assignment can be found in polynomial time al- 
most surely (cf. also [22] for an extension to semirandom formulas). Secondly, 
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one can ask for heuristics that can refute a k-SAT instance, i.e. find a certificate 
that no satisfying assignment exists; of course, in the worst-case this problem 
is cONP-complete. In this paper, we deal with the second problem. More pre- 
cisely, we present strong refutation heuristics, i.e. heuristics that certify that no 
assignment satisfying considerably more than the trivial (1—2~*)-fraction of the 
clauses exists. One motivation for studying this problem is the relationship be- 
tween the existence of strong refutation heuristics and approximation complexity 
pointed out by Feige [8]. 

In order to analyze a heuristic rigorously, we need to specify on which type of 
instances the heuristic is supposed to work properly. In this paper, we consider 
a standard model of random instances of MAX k-SAT. Let V = {21,...,2n} 
be a set of n boolean variables. Then, there are (2n)* possible k-clauses over 
the variables V. If 0 < p < 1, then we let Formn,k,p be a random set of k- 
clauses obtained by including each of the (2n)* possible clauses with probability 
p independently. Hence, the expected number of clauses in ForMmn,k,p is m = 
(2n)*p. 

The combinatorial structure of random k-SAT formulas has attracted con- 
siderable attention. Friedgut [12] has shown that Formņn,k,p exhibits a sharp 
threshold behavior: there exist numbers ck = ck(n) such that Form, ,k,p is satis- 
fiable almost surely if m < (1 — £)ckn, whereas Formn,k,p is unsatisfiable almost 
surely ifm > (1+ €)c,n. The asymptotic behavior of c, as k — oo has been 
determined by Achlioptas and Peres [3] (building on the work of Achlioptas and 
Moore [1]). Moreover, a simple first moment argument shows that the maximum 
number of clauses of Form,,,,) that can be satisfied by any assignment is at 
most (1 — 2* + o(1))m as m/n — oo. More precise results have been obtained 
by Achlioptas, Naor, and Peres [2]. 

Various types of resolution proofs for the non-existence of satisfying as- 
signments have been investigated on Formy,k,»- Ben-Sasson [5] has shown that 
tree-like resolution proofs have size exp(Q(n/A‘/(*-?)*©)) almost surely , where 
A =n*~'p and 0 < € < 1/2 is an arbitrary constant. Hence, tree-like resolution 
proofs are of exponential length even if the expected number of clauses is n*~1!~° 
(i.e. p = 1/n'*°) for any constant 6 > 0. Furthermore, [5, Theorem 2.24] shows 
that general resolution proofs almost surely have exponential size if p < n~*/2~ 
(6 > 0 constant). 

Goerdt and Krivelevich [17] have suggested a heuristic that uses spectral 
techniques for refuting Form, 4, with p = In(n)’n-? (i.e. the expected number 
of clauses is m = In(n)’n*). No efficient resolution-based refutation heuristic 
is known for this range of p. Removing the polylogarithmic factor, Feige and 
Ofek [9] and (independently) Coja-Oghlan, Goerdt, Lanka, and Schadlich [7] 
have shown that spectral techniques can be used to refute Formn,4,p if p > Cn~? 
for a sufficiently large constant C > 0. Moreover, Feige and Ofek [10] have shown 
that a heuristic that combines spectral techniques with extracting and refuting 
a XOR formula from Formn,3,p can refute Form, 3, for p > Cn-3/2 4.6.77 = 
Cn3/*), This result improves on previous work by Friedman and Goerdt [13], 
and Goerdt and Lanka [16]. We emphasize that in all of the above cases, the 
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values of p to which the refutation heuristics apply exceed the threshold when 
Formn,k,p becomes unsatisfiable almost surely by at least a factor of n{*-?)/?, 
The new aspect in the present paper is that we deal with strong refutation 
heuristics. That is, our aim are heuristics that on input Form, x, almost surely 
certify that not more than a (1 —27* +¢)-fraction of the clauses can be satisfied, 
for any € > 0. This aspect has not (at least not explicitly) been studied in 
the aforementioned references. For instance, resolution proofs cannot provide 
strong refutation. Moreover, the spectral heuristics studied so far [7, 9, 10, 13, 
16, 17] only certify that every assignment leaves a o(1)-fraction of the clauses 
unsatisfied. With respect to MAX 3-SAT, we have the following result. 


Theorem 1. Suppose that p > \n(n)®n-*/?. Let € > 0 be an arbitrarily small 
constant. There is a polynomial time algorithm 3-Refute that satisfies the fol- 
lowing. 


— Correctness: For any MAX 3-SAT instance y, the output of 3-Refute(y) is 
an upper bound on the number of satisfiable clauses. 

— Completeness: If py = Formn,3,p, then 3~Refute(y) < (7 + €)n°p almost 
surely. 


Since the number of clauses of Formn,3,p is ~ 8n°p almost surely, 3 -Refute 
does indeed certify almost surely that not more than a z + £ fraction of the 
clauses can be satisfied. Note that the value of p required for Theorem 1 is by a 
factor of In(n)® larger than that required by the heuristic of Feige and Ofek [10] 
(which does not provide strong refutation). We sketch the proof of Theorem 1 
in Section 3. The following result addresses MAX 4-SAT. 


Theorem 2. Suppose that p > con~* for a sufficiently large constant co > 0. 
There is a polynomial time algorithm 4-Refute that satisfies the following. 


— Correctness: For any MAX 4-SAT instance y, the output of 4~Refute(y) is 
an upper bound on the number of satisfiable clauses. 

— Completeness: Ify = Formn,4,p, then almost surely 4-Refute(p) < 15ntp + 
ein a/p. where cı > 0 is a constant. 


4-Refute almost surely certifies that not more than a 4# + O((n,/p)~') 


fraction of the clauses can be satisfied. The second order term O((n,/p)~*) gets 
arbitrarily small as n*p grows. Theorem 2 applies to the same range of p as the 
best previously known refutation heuristics [7,9] for 4-SAT, but provides strong 
refutation. We describe the heuristic for Theorem 2 in Section 2. 

The algorithms for Theorems | and 2 build on and extend the techniques 
proposed in [7,15]. For instance, 4-Refute constructs several graphs from the 
input formula y = Form,y,4,,. To each of these graphs,4-Refuteapplies a sub- 
routine that tries to certify that the graph has “low discrepancy”; i.e. every set 
of vertices spans approximately the expected number of edges. This subroutine 
in turn relies on computing the eigenvalues of a certain auxiliary matrix. Finally, 
if all graphs have passed the discrepancy check, then we conclude that the input 
formula y does not admit an assignment that satisfies more than 15n4p+¢; n/p 
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clauses. The MAX 3-SAT algorithm for Theorem 1 proceeds similarly, but is a 
bit more involved. Though in contrast to [7,15] we obtain strong refutation 
heuristics, the algorithms and the proofs in the present paper are simpler. 

We point out that the approach in [15] can be used to obtain a heuristic 
that almost surely certifies that at most a (f + €)-fraction of the clauses of 
Formn,3,p can be satisfied, though this issue is not addressed explicitly in that 
paper. However, the methods in [15] only seem to apply to somewhat bigger 
values of the clause probability p (namely, p > n~3/2+*, § > 0 fixed) than those 
addressed in Theorem 1. 

The techniques that the algorithms 3-Refute and 4-Refute rely on yield 
heuristics for a variety of further hard computational problems, e.g. for hyper- 
graph problems. Recall that a k-uniform hypergraph H consists of a set V(H) 
of vertices and a set E(H) of edges. The edges are subsets of V(H) of cardinal- 
ity k. An independent set in H is a set S C V(H) such that there is no edge 
e € E(H) with e C S. The independence number a(H) is the number of ver- 
tices in a maximum independent set. Moreover, H is called «-colorable, if there 
exists x independent sets S;,...,5, in H such that S;U---US, = V(H). The 
chromatic number x() is the least integer x > 1 such that H is «-colorable. 

In analogy with the Formn,k,p model of random k-SAT instances, there is 
the Hn x,p-model of random k-uniform hypergraphs: the vertex set of Hn,k,p 1S 
V = {1,...,n}, and each of the c) possible edges is present with probability 0 < 
p < 1 independently. Krivelevich and Sudakov [21] have solved the combinatorial 
problem of determining the probable value of the independence number and of 
the chromatic number of random hypergraphs. The following two theorems deal 
with the algorithmic problem of refuting that a 3-uniform hypergraph has a large 
independent set, or that a 4-uniform hypergraph is K-colorable. We sketch the 
heuristics for Theorems 3 and 4 in Section 4. 

Theorem 3. Lete > 0 be arbitrarily small but fixed. Suppose that p = f /n/?, 
where nên < f = o(n!/?). There is a polynomial time algorithm 3-RefuteInd 
that satisfies the following. 


— Correctness: If H is a 3-uniform hypergraph, then 3-RefuteInd(H) either 
outputs “œ is small” or “fail”. Ifthe answer is “ais small”, then a(H) < en. 

— Completeness: On input H = Hn, 3,p,3-RefuteInd(H) outputs “œ is small” 
almost surely. 


Theorem 4. Let « > 2 be an integer. Suppose that p > coxtn™? for some suffi- 
ciently large constant co > 0. There is a polynomial time algorithm 4-RefuteCol 
that satisfies the following. 


— Correctness: If H is a 4-uniform hypergraph, 4-RefuteCol(A) either outputs 
“not Kk-colorable” or “fail”. If the answer is “not K-colorable”, then x(H) > 
K. 

— Completeness: On input H = Hn,4,p, the output of 4-RefuteCol(A) is “not 
k-colorable” almost surely. 
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In Section 2.2 we present the heuristic for Theorems 2. The main tool is a 
procedure for certifying that a random bipartite graph is of low discrepancy. 
This procedure is the content of Section 2.1. 


2.1 Discrepancy in Random Bipartite Graphs 


Throughout, we let Vj = {vj,...,Un} and Vo = {wy,...,wn} be two disjoint 
sets consisting of n labeled vertices each. We consider bipartite graphs G with 
bipartition (V1, V2), i.e. the vertex set of Gis Vi U Vo, and all edges of G have 
one vertex in V;, and one in Vz. If S1 C Vi and S2 C Va, then we let Eg(S;, S2) 
denote the set of edges in G that connect a vertex in Sı with a vertex in Sz. 
Furthermore, B,,,, denotes a random bipartite graph obtained by including each 
possible edge {v;, wj} with probability p independently. The aim in this section 
is to prove the following proposition. 


Proposition 5. Suppose that np > co for some sufficiently large constant co > 
0. There is a polynomial time algorithm BipDisc and a constant cı > 0 such 
that the following two conditions hold. 


1. Let G be a bipartite graph with bipartition (Vi, V2). On input G, BipDisc 
either outputs “low discrepancy” or “fail”. If BipDisc(G) outputs “low dis- 
crepancy”, then for any two sets Si C Vi, i = 1,2, we have 


|| 51 ]|S2|p ~ |EB (S1, S2)|] < c1V/]Si]|S2inp + nexp(—np/a). (1) 


2. BipDisc( Bn») outputs “low discrepancy” almost surely. 


If [Sil], |S2| = R(n), then Eq. (1) entails that the number |Ee(S1, S2)| of edges 
from S, to S2 in G deviates from its expectation |S ;||S2|p “not too much”. The 
crucial point is that BipDisc certifies that Eq. (1) holds for all sets $1, So. 

BipDisc is based on computing the eigenvalues of a certain auxiliary matrix. 
Given a graph B with bipartition (Vi, V2), we let A = A(B) = (aj; )i,j=1,....n be 
the matrix with entries a;; = 1 if {vi,w;} € E(B), and a, = 0 if {v;,w;} Z 
E(B). Let J denote an n x n matrix with all entries equal to 1. Then, we let 
M = M(B) = pJ — A(B). Furthermore, let |M|| = sup{||Mé||: € € R”, [él] = 
1} denote the norm of M. On input B, ||M]|| can be computed in polynomial 
time up to an arbitrarily small additive error. The next lemma shows what |M || 
has to do with discrepancy certification. 


Lemma 6. Let B be a graph with bipartition (Vi, V2). Then, for any two sets 
Si C Vi, i = 1,2, the inequality ||Ep(S1,52)| — |Si||Salp| < V|S:[|S2] - || M(B) 
holds. 


Sketch of proof. Let £; be the characteristic vector of 5;, i.e. the 7’th entry of 
€; (respectively £2) is 1 if v; € Sı (respectively w; € S2), and O otherwise. 
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Then ||& |] = /|Si|. Hence, |(Mé2,€1)| < W|S1||S2| - |||]. Moreover, a direct 
computation shows that (Mêz, £1) = |.S;||Salp — |EB (S1, S2)|. Oo 


In the case np > In(n)’/n, one can show that ||M|| < O(,/np) almost 
surely (via the “trace method” from [14]). Hence, in this case, by Lemma 6 
we could certify that (1) holds almost surely just by computing ||M(B,,,)||. In 
the case np = O(1), however, we almost surely have that ||M(B,,p)|| > np, 
ie. |M (Br,p)|| is much too large to give the bound (1) (cf. [20] for a detailed 
discussion). Following an idea of Alon and Kahale [4], we avoid this problem by 
removing all edges that are incident with vertices whose degree is too high (at 
least 1Onp, say). This leads to the following algorithm. 


Algorithm 7. BipDisc(G) 
Input: A bipartite graph G = (V1, Vo, E). 
Output: Either “low discrepancy” or “fail”. 


1. If the number of vertices in G that have degree > 10np is > nexp(—cgnp), 
then output “fail” and halt. Here cp > 0 is a sufficiently small constant. 

2. If the number of edges in G that are incident with vertices of degree > 10np 
is larger than c3n*p exp(—cgnp), where c3 > 0 is a sufficiently large constant, 
then halt with output “fail”. 

3. Let G’ be the graph obtained from G by deleting all edges that are incident 
with vertices of degree > 10np. Let M = M(G’). If ||M|| > ca,/np for a 
certain constant c4, then output “fail” and halt. 

4. Output “G has low discrepancy”. 


The analysis of BipDisc is based on two lemmas. 


Lemma 8. There are constants c2,c3 > 0 such that almost surely B = Bn,p has 
the following properties. 


I. The set S of all vertices of degree > 10np has cardinality < nexp(—eonp). 
2. The number of edges in B that are incident with at least one vertex in S is 
< c3n*pexp(—conp). 


Lemma 9. There is a constant cg > 0 such that almost surely the random 
bipartite graph B = Bn p enjoys the following property. Let B’ be the graph 
obtained from B by deleting all edges that are incident with vertices of degree 
> 10np in B. Then, ||M(B')|| < c4,/np. 


Lemma 8 follows from a standard computation. The proof of Lemma 9 is 
based on estimates on the eigenvalues of random matrices from [4]. 


Proof of Proposition 5. Let G = Bn», and let S; C V; for i = 1,2. Moreover, 
let S be the set of vertices of degree > 10np in G. Suppose that BipDisc(G) 
answers “low discrepancy”. Then, by Lemma 6, 


| Ea (S1 \ S,S2\5)] = [S51 \ Si]S2\ Sip} < cav |S1]|S2|np. 
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Moreover, because of Step 2 of BipDisc, we have |E¢($1, S2)| — |Ea(S1 \ S, Se \ 
S)| < egn*pexp(—conp). Further, 

[Si |[Salp — [Si \ S||S2\ Slp < 3np|S| < nexp(—conp/2), 
as otherwise Step 1 would have failed. Thus, (1) holds for S1, S2. Finally, by 
Lemmas 8 and 9, BipDisc(B,,,) outputs “low discrepancy” almost surely. O 
2.2 The Refutation Heuristic for 4-SAT 


Throughout this section, we let V = {z1,...,2%n} be a set of n propositional 
variables. Moreover, we assume that n?p > co for a large constant co. 


Table 1. Clause types and unsatisfied clauses in the case of 4-SAT. 


[pe CUR CVT oe CBW 


lizi V to V 23 V ra| F Te l erar aa 


2zi V £a V Tta V Ba 
3|xı V r2 V Ta V T4 


Alay V T9 Vita V Tal f 


olay V fo V £3 V f4 
Oia, V fo V t3 V Ba 


tila V Žo V ŽV wa j 


Sir, V T2 V a VIa 


| 10 T1 V@2V r3 V £E4| | 
11 Ti V To V Ia V Da T 


12/21 V to V g V Ta 


(131 V Ze V Ta V Ta 
W14174 V to V x3 V Tal 
W15|%1 V Zo V a V T4 





16/71 V fo V 3 V a] | 


Let y be a set of 4-clauses over V. To employ the procedure BipDisc from 
Section 2.1, we construct 16 bipartite graphs G),...,G°® from y. Each G™ is 
a graph with bipartition (V;, V2), where V; = V x V x {i} (i.e. Vi, Vo are disjoint 
copies of V x V). Each graph G™ corresponds to one of the 16 possible ways to 
place the negation signs in a 4-clause: in G™, the edge {(2;,,2i,,1), (Lia, Zig, 2)} 
is present iff the clause l; V Uj, V li, V lig is contained in p, where l;, is either 
ri, Or Z;,, according to the negation signs in Table 1. For instance, the edge 
{(24,, Zig, 1), (Zig, Lig, 2)} is in G™ iff the clause z4 V Zi, V Zi, V Zi, occurs in 
p. Thus, each clause of p induces an edge in one of the graphs G®, and each 
edge results from a unique clause. The algorithm for Theorem 2 is as follows. 


Algorithm 10. 4-Refute(y) 
Input: A set yg of 4-clauses over V. 
Output: An upper bound on the number of satisfiable clauses. 


1. If the number of clauses in ọ is larger than 16n4p + n/p, then return the 
total number of clauses in y as an upper bound and halt. 

2. Compute the graphs G® and run BipDisc(G™) for i = 1,...,16. If the 
answer of BipDisc(G™) is “fail” for at least one i, then return the total 
number of clauses in p and halt. 

3. Return 15n*p + cın’,/p, where cı is a sufficiently large consant. 
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Let us first prove that 4-Refute outputs an upper bound on the number of 
clauses that can be satisfied. 


Lemma 11. There is a constant cz > 0 such that the following holds. Let p be 
a set of 4-clauses such that BipDise(G™) answers “low discrepancy” for all 2. 
Then there is no assignment that satisfies more than |p| —n*p + con3,/p clauses. 


Proof. Consider an assignment that sets the variables T C V to true, and F = 
V\T to false. We shall bound the number of edges in the graphs G®™ that 
correspond to unsatisfied clauses. Let A; C Vi and B; C Və be the sets defined 
in Table 1 for i=1,...,16. Then, in the graph G™, the edges corresponding to 
unsatisfied clauses are precisely the A;-B,-edges. Thus, invoking Proposition 5, 
we have the following bound on the number of unsatisfied clauses: 


16 4 
4 iy4—i 
$. lEo, (Ai Bi) > J (4) (FFT p- can Vp) > np — oan VB 
i=l i=0 
where c2,¢3 are suitable constants. g 


Proof of Theorem 2. The correctness follows from Lemma 11. Since by Chernoff 
bounds (cf. [19, p. 26]) the number of clauses in Formn,4,p is at most 16n*p + 
o(n? /p) almost surely, the completeness follows from Proposition 5. o 


3 Random MAX 3-SAT 


While our refutation heuristic for 4-SAT is based on certifying that certain (bi- 
partite) graphs are of low discrepancy, the heuristic for 3-SAT needs to certify 
that a couple of triple systems are of low discrepancy. In Section 3.1, we de- 
scribe the procedure for certifying low discrepancy in triple systems. Then, in 
Section 3.2, we show how to employ this procedure in order to refute MAX 
3-SAT strongly. 


3.1 Discrepancy in Triple Systems 


Let V = {z1,...,£n} be a fixed set of cardinality n. In this section, we consider 
triple systems over V, i.e. subsets SC V x V x V. If Vi, Vo, V3 C V, then we let 
(Vi, V2, V3)s5 signify the set of triples (v1, ve, v3) € S with v; € V; for 7 = 1,2,3. 
Let £ > 0 be a constant. We say that S has low discrepancy with respect to € if 
the following holds for all X G V with en < |X| < (1 — e)n: letting Y= V\X 
and a = |X|/n, we have 


(X, X, X)s| = (1 +0(1))-a* - |S], 
(X, X, Y)sl; (XY, X)sl (YX, X)s| = (1 + o(1)) -a° (1 — æ) - IS], 
(X,Y, Y)sl, IY, X, Y)|, IY, Y, X)s] = (1 + o(1)) - &(1 — a)? - |S], 
IY, Y, Y)s| = (1 + 0(1)) - (1 — a)? - |S}. 
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For 0 < p < 1, we obtain the random triple system S,,,, by including each 
triple in V? with probability p independently. The aim of this section is to prove 
the following propostion. 


Proposition 12. Lete > 0. There is a polynomial time algorithm TripteDisc, 
that satisfies the following conditions. 


— For each triple system S C V? the output of TripleDise,(S) is either “low 
discrepancy” or “fail”. If the output is “low discrepancy”, then S has low 
discrepancy w.r.t. €E. 

— If p > In(n)®n-3/?, then the output of TripleDisc,(Sn,p») is “low discrep- 
ancy” almost surely. 


To certify that the triple system S C V? is of low discrepancy, the algorithm 
TripleDisc constructs three projection graphs Gij, 1 < i < 7 < 3. The vertex set 
of Gi; is V, and the edge {z, y} is present in G;; iff there is a triple (21, 22,23) E€ S 
with x = z; and y = zj, or x = 2; and y = zi. Thus, if S = Sn,p, then the edge 
{x,y} is present in G;; with probability p’ ~ 2np independently of all other 
edges, so that G;; is distributed as a binomial random graph Gn,p. 

We say that a graph G = (V, E) has low discrepancy w.r.t. £ if for all X C V 
of cardinality en < |X| < (1 — e)n we have 


||Ee(X)|- |X| nE] < e|B] A ||Bo(X, V\X)|—2|X|(n—|X])n™ | EI < |E], 


where Eg(X) is the set of edges in G with both vertices in X, and Eg(X,Y) is 
the set of edges in G with one vertex in X and the other in Y. One ingredient 
to the algorithm TripleDisc for Proposition 12 is to certify that the graphs G;,; 
are of low discrepancy. The following lemma provides us with a polynomial time 
algorithm for this problem. 


1/2 


Lemma 13. Let £ > 0. Suppose that p > 1/n There is a polynomial time 


algorithm A that satisfies the following conditions. 


— Correctness: For any graph G = (V,E), the output of A(G) is either “low 
discrepancy” or “fail”. If the output is “low discrepancy”, then G has low 
discrepancy w.r.t. €. 

— Completeness: [fG = Gn, p», then the output of A(G) is “low discrepancy” 
almost surely. 


The proof of Lemma 13 is based on the relationship between graph discrepancy 
and eigenvalues (cf. [6]) and results on the eigenvalues of random symmetric 
matrices [14]. 

In order to certify that the triple system S has low discrepancy, it is, how- 
ever, not sufficient to check that the projection graphs G;; are of low discrepancy. 
Therefore, in addition to the projection graphs, one could consider the product 
graph Gr = (V x V, Ez), which is defined as follows: an edge {(@1, 61), (a2, b2)} 
is in Ex iff there exists a z € V such that there are two different triples 
(a1, @2, 2), (01, 62,z) € S. Note that in contrast to the projection graphs Gij, 
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the product graph G, is not distributed as a binomial random graph (the edges 
do not occur independently). If the projection graphs G;; and the product graph 
G, all have low discrepancy, then S is of low discrepancy as well. 

However, for the values of p in Proposition 12, we do not know a direct way 
to derive bounds on the eigenvalues of the adjacency matrix of the product graph 
(e.g. it seems difficult to apply the methods in [4,9,15]). Therefore, instead of 
dealing with the product graph and its adjacency matrix, we consider the matrix 
A = A(S,p) defined as follows. For 0 < p < 1 and 61, 62,z E V we let Bbibzz = 
Bb, boz(5,P) = —] if (bj, ba, z) € S, and Bite = Bibibaz (9, p) = p/(1 — p), 
otherwise. Then, the n? x n?-matrix A = A(S,p) = (ab, c;,b2c2)(b1,¢1),(ba,c2)€V? 
is given by 


abi ¢1,bec2 = Y Boies - Boo T Boob, 2z . Beas) if (bj, b2) £ (c1, C2), 
zEV 


and abi ci b2ca = O if (b1, b2) = (c1, cg). Since A is symmetric and real-valued, the 
matrix has n? real eigenvalues Ay > ++: > Anz. We let || Al] = max{A;, —A,2} 
signify the norm of A. 

If S c V3, x€ V, and i€ {1,2,3}, then the degree of x in slot i is dzi = 
\{(z1, 22,23) E€ S : z; = x}|. We say that S$ is asymptotically regular if dz = 
(1 + 0(1))n~|S| for all 2,1. Equipped with these definitions, we can state the 
following sufficient condition for S being of low discrepancy. 


Lemma 14. Let f = pn?/?, and suppose that n° n < f = o(n'/?). If S is a 
triple system that satisfies the following four conditions, then S is of low dis- 
crepancy w.r.t. € > 0. 

1. s=|S| = f -n3/2.(1+0(1)). 

2. S is asymptotically regular. 


3. The three projection graphs of S are of low discrepancy with respect to € > 0. 
4. We have ||A(S,p)|| <In°n- f. 


As by Lemma 13 we can check in polynomial time whether the conditions in 
Lemma 14 hold, we obtain the following algorithm. 


Algorithm 15. TripDisc,(S) 
Input: A set S C V3. Output: Either “low discrepancy” or “fail”. 


1. Check whether Conditions 1—4 in Lemma 14 hold. 
2. If so, output “low discrepancy”. If not, return “fail”. 


In order to prove Proposition 12, it remains to establish that the algorithm 
is complete. A standard application of Chernoff bounds (cf. [19, p. 26]) shows 
that the random triple system S = Sn,p with p as in Proposition 12 satisfies 
Conditions 1-2 in Lemma 14 almost surely. Moreover, the Condition 3 holds 
almost surely by Lemma 13. Thus, it suffices to show that Condition 4 holds 
almost surely. The rather technical proof of the following lemma is based on the 
trace method from [14]. 


Lemma 16. Let f > 1n(n)®, andlet p = fn-*/?. IfS = Sn», then || A(S,p)|| < 
Inž n- f almost surely. 


320 Amin Coja-Oghlan, Andreas Goerdt, and André Lanka 


3.2 The Refutation Heuristic for 3-SAT 


Let y be a set of 3-clauses over the variable set V = {z1,..., £n}. To ap- 
ply the procedure TripDisc from Section 3.1, we construct 8 triple systems 
SO, ...,S® c V3 from y, each corresponding to one of the 8 possible ways 
to set the negation signs in a 3-clause. In the triple system S™, the triple 
(Ziz, Ziz, Zia) € V? is present iff the clause l; V lia V l;i, occurs in y, where 
either l; = 2;, or l, = Z, according to the negation signs for the clause types 
(where clause types are defined similarly as in Section 2.2). 


Algorithm 17. 3-Refute(y, €) 
Input: A set p of 3-clauses over V. 
Output: An upper bound on the number of satisfiable clauses. 


1. Compute the triple systems S$ and run TripDisc, ).(S) fori=1,...,8. 
If the output is “fail” for at least one 2, then return the total number of 
clauses in y as an upper bound and halt. 

2. Return (7 + €)n3p. 


The proof of Theorem 1 relies on a similar argument as the proof of Lemma 11. 


4 Hypergraph Problems 


Let H = (V, E) = Hn,4,p be a random 4-uniform hypergraph with vertex set V = 
{1,...,n}. Let « be an integer, and suppose that p > cox*n~? for a sufficiently 
large constant co. The algorithm 4-RefuteCol for Theorem 4 is randomized. On 
input H, the algorithm obtains a set S C V4 of ordered 4-tuples as follows (recall 
that the edges E are not ordered). If e = {r1, x2, £3, £4} € E, then there are 4! = 
24 possibilities to order the vertices 21, £2, £3, £4. Let Te) be the set of the 24 
possible ordered tuples. Letting pp = 1 — (1 — p)!/?4, we choose the set Ø # Xe C 
T (e) of tuples that we include into S to represent e according to the distribution 
P(X) = pi*-!(1 —po)24-!Xelp-1. Thus, each edge e € E gives rise to at least one 
tuple in S. The choice of the sets Xe is independent for all e € E. Furthermore, 
we include each tuple (£1, £2, £3, £4) € V4 such that |{21, £2, £3, £4}| < 4 into S 
with probability pọ independently. A trite computation shows thatif H = Hn,4,p, 
then the resulting set S = S(H) of 4-tuples is distributed so that every possible 
4-tuple in V4 is present with probability po independently. 

Let Vi = V x V x {1}, V2 = V x V x {2} be two disjoint copies of V. Having 
computed S = S(H), 4-RefuteCol constructs a graph G with bipartition (V1, V2) 
in which the edge {(21, £2, 1), (£3, 24, 2)} is presentiff (£1, £2, £3, £4) € S.If H = 
Hy,4,p, then G is a random bipartite graph B,,2 po- Tothis graph G, 4-RefuteCol 
applies the procedure BipDisc. If BipDisc answers “low discrepancy’, then 4- 
RefuteCol answers “His not «-colorable”. Otherwise, the output is “fail”. 

The heuristic 3-RefuteInd(H) for Theorem 3 transforms the hypergraph H 
into a triple system and applies TripleDisc. 
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Abstract. It is exponentially unlikely that a sparse random graph or 
hypergraph is connected, but such graphs occur commonly as the giant 
components of larger random graphs. This simple observation allows us 
to estimate the number of connected graphs, and more generally the num- 
ber of connected d-uniform hypergraphs, on n vertices with m = O(n) 
edges. We also estimate the probability that a binomial random hyper- 
graph Ha(n, p) is connected, and determine the expected number of edges 
of Ha(n,p) conditioned on its being connected. This generalizes prior 
work of Bender, Canfield, and McKay [2] on the number of connected 
graphs; however, our approach relies on elementary probabilistic meth- 
ods, extending an approach of O’Connell, rather than using powerful 
tools from enumerative combinatorics. We also estimate the probability 
for each t that, given k = O(n) balls in n bins, every bin is occupied by 
at least t balls. 





1 Introduction and Results 


A d-uniform hypergraph H consists of a set V of vertices and a set E of edges, 
which are subsets of V of cardinality d. Thus, a 2-uniform hypergraph is just a 
graph. A vertex w is reachable in H from a vertex v if either v = w or there is 
a sequence e€1,...,e% of edges such that v € e1, w E ek, and e; N ei+1 Æ Ô for 
i = 1,...,k—1. Reachability in H is an equivalence relation, and the equivalence 
classes are the connected components of H. We say that H is connected if there 
is only one component. 

Connectedness is perhaps the most basic property of graphs and hypergraphs, 
and estimating the number of connected graphs or hypergraphs with a given 
number of vertices and edges is a fundamental combinatorial problem. In most 
applications, one is interested in asymptotic results, where the number of ver- 
tices/edges tends to infinity. The main result of this paper is a formula for the 
asymptotic number of connected d-uniform hypergraphs; or, equivalently, an es- 
timate of the probability that a random d-uniform hypergraph is connected. 
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Our results hold up to a constant multiplicative factor. We study both a model 
Ha(n,m) in which the number of vertices and edges is fixed, and a binomial 
model Hu(n,p) in which each possible edge appears with probability p; in the 
latter case we also calculate the expected number of edges conditioned on the 
graph being connected. Furthermore, we obtain a simple algorithm for generating 
a connected hypergraph uniformly at random. 

We obtain these results using a new probabilistic approach. Rather than using 
powerful techniques of enumerative combinatorics such as generating functions 
and complex analysis, our calculations are for the most part elementary, and 
rely on the fact that a connected (hyper)graph of a given order and size is 
likely to occur as the giant component of a larger one. We believe that this 
approach is of interest in its own right, and can be applied to a number of 
problems in combinatorics and random (hyper)graph theory; to illustrate this, 
we also calculate the probability that every bin in a balls-and-bins experiment 
is occupied by at least t balls. 

For the special case of graphs, i.e., for d = 2, the results we present in this 
paper (and, in some cases, stronger results) are already known. Compared to 
graphs, little is known about d-uniform hypergraphs where d > 3, although 
several papers deal with their component structure (cf. Section 2 for details). 


Given a hypergraph H = (V,E), its order is its number of vertices |V|, and 
its size is its number of edges |E|. The degree dy(v) of a vertex v € V is the 
number of edges e € E such that v € e. Note that if H is d-uniform, its average 
degree is d|E|/n, Throughout, V = {1,...,n} is a fixed set of n labeled vertices 
and d > 2 is a fixed integer. We let Ca(n, m) signify the number of connected 
d-uniform hypergraphs of order n and size m. Observe that if a hypergraph is 
connected, then its average degree is at least d/(d — 1). 

Two natural random models present themselves. In Ha(n, m), we select one 


of the ((a)) sets of m possible edges at random. In Ha(n, p), each of the (3) edges 
appears independently with probability p, in which case the expected number 
of edges is p() ~ pn*/d! and the expected degree is pos) ~ pnt—3/(d — 1)! 
Thus we will compare Hga(n,m) where m = cn/d with Hg(n,p) where p = 
c(d — 1)! /n4-!, While these two models are interchangeable in many respects, 
with respect to large deviations they are not; in particular, as we will see, their 
probabilities of connectedness differ by an exponential factor. 

The following theorem determines the asymptotic probability of connect- 
edness of Ha(n, m), or equivalently the number of connected d-uniform hy- 
pergraphs, up to a constant factor in the regime where the average degree is 


ce > d/(d—1). 


Theorem 1. Letc > d/(d—1) be a constant independent of n, and let m = cn/d. 
Let a = a(c) be the unique solution in (0,1) of the equation 
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Fig. 1. Left figure: the functions $3(c) (black) and ¥3(c) (gray) in the range 1.5 < c < 3. 
Right figure: the functions -y3(c) (black) and c++ c (gray) in the range 0.5 < ce < 5. 


and set ale) = at~ (1 — a)O/4)-1 (1 — (1 — a)4)°/4 . Then the probability that 
Ha(n,m) is connected is O(@q(c)"), and so Ca(n,m) = O (Sa(c)” ((2)) } (as 


n — o0). 


Theorem 1 addresses the combinatorial problem of estimating the number of 
connected hypergraphs. But there is also a corresponding algorithmic problem: 
can we sample a connected hypergraph of a given order and size uniformly at 
random? We answer this question in the affirmative. 


Theorem 2. Let c > d/(d — 1), and let m = en/d. There is a randomized 
algorithm that samples a connected hypergraph of order n and size m uniformly 
at random in expected polynomial time. 


Our next theorem gives an result analogous to Theorem 1 for Ha(n, p). 


Theorem 3. Let c > 0 be a constant, and let p = c(d —1)!/n¢~1. Leta be the 
unique solution in (0,1) of the equation 


| 


1 — a = exp (~< al (2) 


a 


Ha(n,p) is connected is O(Wa(c)"). For d = 2, Yo(c) = 1 — exp(—c). 


and set Uq(c) = a (1 — a)/%)-1 exp (s . tewtayee | . Then the probability that 


To illustrate Theorems 1 and 3 we plot the functions $3(c) and %3(c) in Fig- 
ure 1. In fact 3 is larger than $3, i.e., Ha(n, p) is exponentially more likely to be 
connected than Hy(n,m) even when the two have the same (expected) average 
degree. The reason for this is that in Hg(n,p) the number of edges is a random 
variable; we can think of Ha(n,p) as first choosing a number of edges m’ ac- 
cording to the binomial distribution Bin((%), p), and then choosing a Ha(n, m’). 
Thus Ha(n, p) can boost its probability of being connected by including a larger 
number m’ of edges. Indeed, if we condition on Ha(n,p) being connected, the 
conditional expectation of the number of edges will be significantly larger than 
p(). Our next theorem quantifies this observation. 
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Theorem 4. As in Theorem 3, let c > 0 be a constant independent of n, let 
p = c(d ~—1)!/n4~1, and leta be the unique solution in (0,1) of Equation (2). 


Set 
Ya(c) =€ (=) ; 


ad 
The expected number of edges of Ha(n,p) conditioned on the event that Ha(n, p) 


is connected is nya(c)/d + o(n), so the expected average degree is ya(c) + o(1). 
For d=2, yale) = c» (ef + 1)/(e® — 1) =c- coth(c/2). 


The proof of Theorem 4 consists of maximizing the product of g(¢ = dm/n)” 
times the binomial distribution Bin((%), p). Figure 1 shows -y3(c). As c — oo, 
yafe} — c, since connectedness becomes a less unusual condition as c increases. 
As c— 0, ¥-(d) — d/(d — 1), the minimum average degree required for connec- 
tivity. 

One ingredient of the proofs of Theorems 1—4 is the component structure of 
random hypergraphs Ha(n,m) and Ha(n,p). We say that Hg(n,m) or Ha(n, p) 
has a certain property with high probability (w.h.p.) if the probability that the 
property holds tends to 1 as n — oo. Suppose that the average degree c = 
dm/n (resp. ¢ = pn4—1/(d — 1)! is a constant greater than 1/(d — 1). Then, 
just as for graphs [6], w.h.p. there is a unique giant component of order R(n) 
in Ha(n, p), and all other connected components are of order O(ln n) [7,13]. To 
prove Theorems 1—4, it is crucial to have a rather tight estimate on the order 
and size of the giant component. 

We say that a random variable X defined on Ha(n, m) or Ha(n, p} is concen- 
trated in width y about z if for each € > 0 there is a number C(e} independent of 
n such that Pr||X — z| > C(e)y] < £. Further, we write f(n) = O(g(n)) if there 
is a constant c > 0 such that |f(n)| < |g(n)| In‘ n for all sufficiently large n. 


Theorem 5. Suppose that c > 1/(d—1). Let a = a(c) be the unique solution in 
(0,1) of the equation 


1 — a = exp (—c (1 — (1 —a)*"*)) 


and let b = b(c) = 1 — (1 —a)*. Let m = cnjd and let p = e(d — 1)! /n®7t, 
Then, the expected order (resp. size) of the giant component of Ha(n, m) and 
of Ha(n,p) is an + O(1) (resp. bm + O(1)). Moreover, the order (resp. size) of 
the giant component of both Ha(n,m) and Ha(n,p) is concentrated in width yn 
about an (resp. bm). 


Note that setting d = 2 in Theorem 5 recovers the classic result 1 — a = e~ 
for random graphs [6]. To further illustrate the applicability of our approach, we 
solve a type of occupancy problem. Suppose k balls are placed randomly in n 
bins; the following theorem estimates the probability q:(k,n) that every bin is 
occupied by at least t balls. 


Theorem 6. Lett > 1 be an integer, letr >t be a constant independent of n, 
and let k = rn. Let a = a(r) and B = f(r) be the unique solutions in (0,1) to 
the pair of equations 
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C e-72/8 (raj py Laps se 72/B( ae 
—_ 


7=0 j=0 


œ&=1— 


Setri =r. =f iza, and let A = X(r) be the unique solution of the equation 


Pa i Ge Gai = =r] Sao i à unless t = 1, in which case set A= 1. Then set 


1—1 
bii i 


ast, Seeds tir(i-4)-1 -r 1 
g(r) =at-"(1—a)atrG-A)-1 | rite ‘IIs, 


where b; = sea Then, qe(k, n) = O(¢(r)”) as n —> o. 
£=0 i 
For t = 1 in particular, Theorem 6 gives œ = 1 — e7™"®%, 2 = 1, and r; = 0, 
\ = 1,and bo = 1. Then qı(k, n) = O ((at7"e"(*-0)”) The proof of Theorem 6 
is similar to the proofs of Theorems 1 and 3 and is omitted. 


2 Related Work 


The asymptotic number of connected graphs with a given number of edges and 
vertices was determined by Bender, Canfield, and McKay [2]. Their proof is 
based on generating functions, and is stronger than Theorem 1 in that it de- 
termines the number of connected graphs up to a multiplicative factor 1 + o(1) 
instead of a constant. The same authors have determined the probability that 
a random graph G,,,, is connected as well as the expected number of edges of 
Gn,p Conditioned on connectedness [3], corresponding to the case d = 2 of The- 
orems 3 and 4; however, they give the asymptotic distribution of the number of 
edges conditioned on connectedness, not just its expectation. An algorithm for 
generating random connected graphs (the case d = 2 of Theorem 2) immediately 
follows from the results in [2, 3]. Pittel and Wormald [12] presented an alternative 
approach based on enumerating graphs of minimum degree 2 to derive slightly 
improved versions of the results in [2]. Among other things, O’Connell [11] has 
estimated the probability that a random graph Gz,» is connected using a prob- 
abilistic approach. However, the result in [11] is somewhat less precise than [3]. 
Using large-deviation methods from statistical physics, Engel, Monasson, and 
Hartmann [4] have investigated exponentially unlikely component structures of 
random graphs, including connectedness and, more generally, a specified number 
of components per vertex. 

The asymptotic order and size of the giant component of random graphs (the 
case d = 2 of Theorem 5) has been known since the pioneering work of Erdés 
and Renyi (cf. [6] for a unified treatment). Furthermore, Barraez, Boucheron, 
and Fernandez de la Vega [1] give a description of the asymptotic distribution 
of the order and size of the giant component of random graphs. 

For d-uniform hypergraphs, d > 3, on the other hand, little seems to be 
known. Karonski and Luczak [8] determined the number of connected hyper- 
graphs of order n and size m = n/(d — 1) +k where k = o(Inn/InInn) (i.e., 
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just above the number of edges necessary for connectedness) up to a factor of 
1+o(1) via purely combinatorial techniques. Since Theorem | addresses the case 
m = cn/dfor c > d/(d—1), our results and those of [8] are incomparable. 

Schmidt-Pruzan and Shamir [13] have shown that in a very general model of 
random hypergraphs a phase transition phenomenon occurs: there is a certain 
average degree c* such that for all e < (1~é)c* the largest component of Ha(n, m) 
with m = cn/d has order O(inn), whereas for c > (1 + €)c* there is a unique 
giant component of order {2(n). In the case of d-uniform hypergraphs, the critical 
average degree is c* = 1/(d—1). Karonski and Luczak [7] have studied this phase 
transition in greater detail. To the best of our knowledge, the expected order 
and size of the giant component as given in Theorem 5 have not been stated 
before (although our proof relies on standard methods). 

With regards to the occupancy problem, in the case t = 1, more precise results 
for qı(k, n), can be obtained from asymptotic formulas for Stirling numbers of 
the second kind due to Temme [14] and Moser and Wyman [10]. However, these 
results rely on generating functions and complex analysis, while our approach 
is elementary. For ¢ > 1, to the best of our knowledge the results stated in 
Theorem 6 are new. 


3 Techniques and Outline 


While most of the previous work relies on rather heavy machinery from enumer- 
ative combinatorics, our approach is quite different and purely probabilistic, and 
yields shorter and simpler proofs. For instance, to prove Theorem 1 the basic 
idea is as follows. Suppose that m = en/d for some c > 1/(d — 1), and let a,b 
be as in Theorem 5. Then, by Theorem 5, w.h.p. there is a unique giant com- 
ponent in Ha(n, m), which—conditioned on its order and size—is a uniformly 
random connected hypergraph. As we shall see in Section 5, we can express the 
number Calan, bm) of connected hypergraphs of order an and size bm in terms 
of the probability y(an, bm) that the giant component of Ha(n, m) has precisely 
order an and size bm. Then, in Section 6, we reduce the problem of computing 
x{an,bm) to a balls-and-bins problem. Section 4 contains the proof of Theo- 
rem 5. The algorithm for Theorem 2 drops out of our proof of Theorem 1 in 
Section 6 immediately. 

Our approach is related to that of O’Connell [11], but goes beyond it consid- 
erably. O’Connell shows that Pr[Gn p is connected] = W2(np)"t°™ by observing 
that w.h.p. the giant component of Gn p has order an + o(n) where 1 — a = e~** 
(the d = 2 case of Theorem 5). However, due to the o(n) error term, in [11] it 
is necessary to prove that a certain rate function is continuous. This proof relies 
heavily on the independence of the edges in Gn,p, and therefore is not easy to 
adapt to the Gn,m model. By contrast, we do not need to work with rate func- 
tions, because we estimate the probability that the giant component of G,, |, has 
exactly order an. In effect, we determine Pr[G,,,, is connected] up to a constant, 
rather than subexponential, factor. 
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4 The Giant Component of Ha(n,m) and Ha(n, p) 


To estimate the expected order and size of the giant component of a random hy- 
pergraph Ha(n, p), we establish an analogy between a Galton-Watson branching 
process with a suitable successor distribution and exploring the components of 
a random hypergraph. For graphs, a similar approach is well known (cf. [6]). To 
estimate the expected order/size of the giant component of a hypergraph up to 
an additive error of O(1), we need to refine this approach slightly. 

Let us first describe the branching process. During the process there are two 
kinds of “organisms”: living and dead ones. In the beginning, there is only one 
organism, which is alive. In the 7’th step, a living organism is chosen, produces 
a number Z; of children, and dies. The Z;’s are independent and is distributed 
as a multiple (d — 1) - Po(c) of a Poisson random variable with mean c. We say 
that the process dies out if at some step 2 the last living organism dies without 
producing any offspring. General results [5, p. 297] imply that the probability 
p that the process dies out is the unique solution in (0, 1) of the equation p = 
exp (c(p?-? — 1)). 

Now, for random hypergraphs H = Ha(n, p), consider the following process, 
which explores the connected component of a vertex v. During the process, the 
vertices of H are either dead, alive, or neutral. Initially, only v is alive, and all 
other vertices are neutral. In each step, a living vertex w is chosen. We investigate 
all edges e of H that contain w, at least one neutral vertex, but no dead vertex 
(since a dead vertex indicates that that edge has already been explored). All 
neutral vertices contained in such edges are made live, and w dies. Let Z, be 
the number of vertices made live from w. When there are no living vertices left, 
the set of dead vertices is precisely the connected component of v. 

Recall that a random variable X dominates Y if Pr[Y < t] < Pr[X < t] for 
all t. Let Bin(N, p) denote a binomial random variable with parameters N and 
p. Then, on the one hand, Zw is dominated by (d — 1)Bin( (ap) since the 


degree of w in Ha(n, p) is distributed as Bin( (721), p). On the other hand, Zw 


dominates the following random variable Sx: with probability 1—O(1/n), S; has 
distribution (d — 1)Bin((7— 1), p), and with the remaining probability Sẹ = 0. 
Using these estimates, a similar argument as in [6, proof of Theorem 5.4] yields 
the following. 


Lemma 7. Leta,b be as in Theorem 5. The expected number of vertices outside 
the giant component is (1—a)n+O(1), and the expected number of edges outside 
the giant component is (1 — b)m + O(1). 


Lemma 7 establishes the first part of Theorem 5. In order to show that the 
order and size of the giant component of Hg(n,p) are concentrated about their 
means, we show that both have variance O(n), and apply Chebyshev’s inequality. 
A similar approach works for Ha(n, m). 


5 The Number of Connected Hypergraphs 


Suppose that m = cn/d forsome c > 1/(d—1). Following the outline in Section 3, 
in this section we reduce the problem of counting connected hypergraphs to 
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estimating the probability that the giant component of Hg(n,m) has a given 
order and size. Let a,b be as in Theorem 5, and set 


en JACQ. o 


any, —1 
Let calan, bm) = Calan, bm) (ÇE )) be the probability that Hg(an, bm) is con- 
nected. Then, ~(an, bm) - calan, bm) is the expected number of components of 
order an and size bm in Ha(n,m). Furthermore, if H is a hypergraph, then we 
let M(H) (resp. M(H)) denote the order (resp. size) of the component with 
the most vertices (resp. edges). Then letting x{z,y) be the probability that 
H = Ha(n,m) satisfies M(H) = z and M(H) = y, we obtain 


x(an, bm) < wan, bm) : calan, bm) (4) 


(ford = 2, a similar inequality has been derived in [9]). Furthermore, ~(an, bm)- 
calan, bm) -PrlN(Ha(n — an, m —bm)) < an] is the probability that in Hg(n, m) 
there is precisely one component of order an and size bm. Hence, 


(a, b) - ca(an, bm) - Pr(N(Ha(n — an,m — bm)) < an] < x(an,bm). (5) 
Let us first get rid of the term PrN (Ha(n — an,m — bm)) < an]. 
Lemma 8. PrlMV/(Ha(n — an,m — bm)) < an] ~ 1. 


Lemma 8 follows from the analogy between random hypergraphs and branch- 
ing processes discussed in Section 4. Lemma 8, (4) and (5) yield 


x(an, bm) ~ oan, bm) - calan, bm) . (6) 


Thus, we have reduced the problem of computing ca(an, bm) to the problem of 
computing x(an, bm). The following lemma, which we prove in Section 6, does 
this within a multiplicative constant. 


Lemma 9. y(an, bm) = O((nm) 1/2). 


Combining Lemma 9 and (6) gives ca(an, bm) = O ((nm)—'/? (an, bm)—*). 
Let v = an and u = bm. Expanding ~(an, bm) using Stirling’s formula yields 


_ (abl -a(l — binm)? aa 
any = o 


i (Ben) ” ( Qa- om) B 
(7"\m (Umann 


Let ¢ = du/v, i.e., the average degree of the giant component if it has order v 
and size 4. Plugging the relation (1 — a)* = 1 — b into (7), we obtain 


O(ca(v, u)) = (als fia) se a)4)/*) 


(1 = a)(t-a)" (7) 





V 





330 Amin Coja-Oghlan, Cristopher Moore, and Vishal Sanwalani 


Finally, using 1 — a = exp(—c(1—(1—a)*')) and b = 1 — (1 — a)? from 
Theorem 5 and writing c = dm/n = a/b, we get 


1 — (1 — a)®-! 
L — a = exp (—Ca teen 
thereby proving Theorem 1. 
To prove Theorem 3, we write an equation analogous to (3). If cg(n, p) is the 
probability that Ha(n,p) is connected, then the expected number of connected 


Th 


components of order x is (")(1 — p)(a)—(a)- ("a") -¢q(z,p) . We also have the 
following lemma, analogous to Lemma 9. 


Lemma 10. Let a be as in Theorem 5. The giant component of Ha(n,p) has 
order an with probability @(n~\/2), 


Theorem 3 then follows from Lemma 10 and algebraic manipulations similar to 
those for Theorem 1. 

To prove Theorem 2, we observe that Lemma 9 yields a very simple random- 
ized algorithm for generating a uniformly random connected hypergraph of order 
v and size p = ¢v/d for constant ¢ > d/(d — 1) in expected polynomial time. 
First, fix n, m,c such that a(c)n = v and b(c)m = u where a = a(c), b = b(c) 
are as in Theorem 5. Then construct a random hypergraph Ha(n, m) and find 
its giant component C. Since C is uniformly random conditioned on its order and 
size, we output it if it has order v and size yu, and if not we repeat. By Lemma 9, 
this simple rejection sampling approach succeeds in expected time polynomial. 


6 The Probability of Getting a Giant Component 
of a Given Order and Size 


Lemma 9 is a rather uncommon statement about random hypergraphs, since it 
concerns the probability that the order and size of the giant component attain 
exactly one particular value. A more common type of result (which is, of course, 
interesting in its own right) would be to characterize the asymptotic distribu- 
tion of e.g. N(Gn,m). For instance, in [1] it is shown that V(Grjm)//n has an 
asymptotically normal distribution. However, as this statement allows for addi- 
tive errors of size o(,/n), it does not imply Lemma 9. In this section, we shall 
prove that the following lemma on fluctuations in the classical balls-and-bins- 
problem implies Lemma 9. (Lemma 10 can be derived analogously.) 


Lemma 11. Suppose that we throw r ballsinto n bins uniformly at random and 
independently, where r = pn for some constant p > 0. Let Z denote the number 
of empty bins. Then for each C > Q there is a 6 > 0 such that for all z satisfying 
|z — E(Z)| < C\/n we have Pr{Z = z] > ôn. 


There are (at least) two ways to prove Lemma 11. On the one hand, one can 
derive the lemma from rather involved combinatorial results such as [14]. On 
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the other hand, there is also a simple probabilistic argument using sums of 
independent random variables (details omitted). 

Here is the basic idea for how to employ Lemma 11 to prove Lemma 9. We 
expose the edges of the random hypergraph Ha(n,m) in two stages. First, we 
throw in mı = (1—¢€)m random edges, thereby obtaining Hı = Ha(n, mı). Here 
£ > 0 is a sufficiently small constant. Let C denote the giant component of H4. 
Then, we obtain Ha(n, m) by adding a set F of mo = em further random edges 
to Hı. Let Fpa be the set of all edges in F that have d—1 vertices in C and whose 
d’th vertex either is an isolated vertex of H, or belongs to an isolated edge of H4. 
Then, we first add the edges in F\ Fhan to Hi. Finally, we add the edges in Fay, 
thereby attaching isolated vertices and edges to the giant component. We shall 
model the number of isolated edges/vertices that we attach as a balls-and-bins 
experiment, the edges in Fhan being the balls, and the isolated edges/vertices of 
H; being the bins. By Theorem 5, N(Ha(n, m) = Hı + F} and M(H, + F) are 
concentrated in width y/n about their means an, bm. Thus, in order to achieve 
e.g. that the order of Hı +F is precisely an, we need that the number of vertices 
attached to the giant component via Fy.) is precisely an — N(Hi + F \ Fhan), 
and indeed Lemma 11 estimates the probability that this is the case. We need to 
consider both isolated vertices and edges, because we wish to prove a statement 
on both the order and the size of the giant component. 

However, there are some details that we have glossed over so far. For instance, 
there could be edges in F that fall into V \ C and “eat up” isolated vertices 
that the edges in Fy); would like to use as bins. Moreover, probably there will 
be edges in F that contain vertices in the giant component of Hı and several 
isolated vertices, etc. Therefore, we need to consider a finer partition of F. First, 
we add those edges that either land inside the giant component of Hı or inside 
its complement entirely: let 


F,={ee Flecc}, h={eeFlecV\C}, m=Mh+At+h. 


Ife is small enough, then w.h.p. all components of Hj2—C will have order O(n n), 
so that C is still the giant component of H2. As a next step, we add 


— the set F3 of edges in F that join a component of order > d of Hz — C toC, 

— the set F4 of all edges e € F that are incident with less than d — 1 vertices 
in C, 

— and the set Fs of edges e € F that share precisely d—1 vertices with another 
edgee #e' EF. 


All edges in F3 U F4 U Fs join vertices in the giant component C of Hə with 
vertices in V \ C. Let H3 = Ha + F; + Fa + Fs, and let C’ be the vertex set 
of the giant component of Hg. Furthermore, let F” be the set of all edges in 
e € F\(FiU--- U F5) that join a component of order 1 or of order d (i.e. an 
isolated edge) of H3 to C’. By the choice of F4 and Fz, F” has the following two 
properties. 


i. Each e € F’ hasd—1 vertices inC CC’. 
ii. For any two edges e,e’ € F’ we have eNC #e' NC. 
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The edges in F” will be our “balls”. Finally, let Fẹ = F \ (F, U---U Fs UF’), and 
H, = H3 + Fe. Then, the edges in Fg connect C with an isolated vertex/edge of 
Hə, but we can’t use these edges as “balls”, because their isolated vertex/edge 
is also connected with C through another edge in F. Observe that all vertices of 
edges in Fe lie inside the giant component of H3, so that the vertex set of the 
giant component of H4 is still C”. 

Let Yı (resp. V2) be the set of isolated vertices (resp. edges) of H4, and set 
Y; = #);. Moreover, let FÍ (resp. Fz) be the set of all e € F” that connect an 
isolated vertex (resp. edge) of H4 with C’, and set Z; = #F/. Finally, let X; 
(resp. X2) be the number of isolated vertices (resp. edges) that the edges in Fj 
(resp. Fz) connect with C’. The following observation is crucial for our argument. 


Proposition 12. If we condition on any specific outcome of (Yi, Y2, Z1, Z2), 
then Xı, X2 are independent. Moreover, the conditional distribution of X; is 
precisely the distribution of the number of non-empty bins when Z; balls are 
thrown into Y; bins uniformly at random and independently. 


Corollary 13. Let u,(Z;) = E(X; | Zi = 2). Conditioned on Zı = zı and 
Z2 = 22, both Xı and Xz are concentrated in width y/n about uı(zı), p2(z2). 


Sketch of proof. Condition on the event that Z4 = 21, Z2 = z2, and Yı = y1, 
Yə = y2. By the proposition, X 1, X29 are independent and distributed as in a balls- 
and-bins experiment. Therefore, a standard application of Azuma’s inequality 
(cf. [6, p. 37]) shows that both X; and X2 are concentrated in width y^. o 

As a next step we prove that the number of balls and bins is of the same 
order of magnitude as n. 


Lemma 14. With probability > 1 — nt? we have Yi, Y2, Zi, Z2 = R(n). 


Observe that the number of vertices that the edges in F” attach to the giant 
component C’ of H4 is Xı + dXz2. Furthermore, the number of edges (including 
the ones in F”) joined to C’ via F’ is Z1 + Z2 + X2. 


Lemma 15. Let C > 0 be a sufficiently large constant. Let Z denote the set of 
all pairs (21,22) such that the following holds: 


Pr[|IN (H4) — an — (u1 (21) + dua(z2))| < C'n and (8) 
IM(H4) — bm —- (zı + Z2 + u2(z2))l < Cyn | LZ. = 215.20 = z2] > 2 i 


Then Pr|(Z, Z2) € Z] > 99/100. 


Sketch of proof. Let 4 > 0 be a sufficiently large constant. On the one hand, by 
Corollary 13, the probability that the number of isolated vertices or the number 
of isolated edges attached to C deviates by more than 4,/n from the expectation 
uı(Zı), #2(Z2) is less than 6. On the other hand, by Theorem 5, the order and 
the size of the giant component of Ha(n, m) are concentrated in width yn. O 


Proof of Lemma 9 (sketch). To prove the lower bound 2((nm)~1/?), condition on 
the following event £1: we have Y1, Y2, Z1, Z2 = R(n) and the outcome (21, 22) of 
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(Z,, Z2) is such that (8) holds. By Lemmas 14 and 15, Pr[&] > 9/10. Moreover, 
by Lemma 15, the conditional probability that H4 satisfies 


IN (Ha) —an—(pi(21)+due(z2))|, |M(H4)—bm-— (z1 +z2+u2(22))| < Cyn (9) 


is at least 1/2. We additionally condition on the event Ez that (9) holds. Then, 
Proposition 12 and Lemma 11 entail 


Prix, +dXo = an—N(H13), 241+Z0+ Xe = bm — M(H4) lE N E2] = @O(nm)~?. 


Furthermore, Pr[&) N €2] > 9/20, thereby proving the lower bound claimed in 
Lemma 9. The (even simpler) proof of the upper bound is omitted. x 
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Abstract. Given two independent weak random sources X, Y, with 
the same length £ and min-entropies bx,by whose sum is greater than 
L + §2(polylog(€/e)), we construct a deterministic two-source extractor 
(aka “blender’’) that extracts max(bx, by) + (bx + by — £ — 4log(1/e)) 
bits which are €-close to uniform. In contrast, best previously published 
construction [4] extracted at most $(bx + by — £—2log(1/e)) bits. Our 
main technical tool is a construction of a strong two-source extractor 
that extracts (bx + by — £) — 2log(1/e) bits which are ¢-close to being 
uniform and independent of one of the sources (aka “strong blender”), 
so that they can later be reused as a seed to a seeded extractor. Our 
strong two-source extractor construction improves the best previously 
published construction of such strong blenders [7] by a factor of 2, ap- 
plies to more sources X and Y, and is considerably simpler than the 
latter. Our methodology also unifies several of the previous two-source 
extractor constructions from the literature. 


1 Introduction 


IMPERFECT RANDOMNESS. Randomization has proved to be extremely useful 
and fundamental in many areas of computer science. Unfortunately, in many 
situations one does not have ideal sources of randomness, and therefore has to 
base a given application on imperfect sources of randomness. 

Among many imperfect sources considered so far, perhaps the most general 
and realistic source is the weak source [29, 4]. The only thing guaranteed about 
a weak source is that no string (of some given length £) occurs with probability 
more than 2~°, where b is the so-called min-entropy of the source. We will call this 
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source (£, b)-weak. Unfortunately, handling such weak sources is often necessary 
in many applications, as it is typically hard to assume much structure on the 
source beside the fact that it contains some randomness. Thus, by now a universal 
goal in basing some application on imperfect sources is to make it work with weak 
sources. The most direct way of utilizing weak sources would be to extract nearly 
perfect randomness from such a source. Unfortunately, it is trivial to see [4] that 
no deterministic function can extract even one random bit from a weak source, 
as long as b < £—1 (i.e., the source is not almost random to begin with). This 
observation leaves two possible options. First, one can try to use weak sources 
for a given application without an intermediate step of extracting randomness 
from it. Second, one can try designing probabilistic extractors, and later justify 
where and how one can obtain the additional randomness needed for extraction. 


USING A SINGLE WEAK SOURCE. A big and successful line of research [26, 24, 4, 
5, 29, 2] following the first approach showed that a single weak source is sufficient 
to simulate any probabilistic computation of decision or optimization problems 
(1.e., problems with a “correct” output which are potentially solved more effi- 
ciently using randomization; That is, all problems in BPP). Unfortunately, most 
of the methods in this area are not applicable for applications of randomness, 
where the randomness is needed by the application itself, and not mainly for 
the purposes of efficiency. One prime example of this is cryptography. For ex- 
ample, secret keys have to be random, and many cryptographic primitives (such 
as public-key encryption) must be probabilistic. The problem of basing a cryp- 
tographic protocol on a single weak random source has only been studied in the 
setting of information-theoretic symmetric-key cryptography. In this scenario, 
the shared secret key between the sender and the recipient is no longer random, 
but comes from a weak source. As a very negative result, McInnes and Pinkas [12] 
proved that one cannot securely encrypt even a single bit, even when using an 
“almost random” (£,@— 1)-weak source. Thus, one cannot base symmetric-key 
encryption on weak sources. Dodis and Spencer [6] also consider the question of 
message authentication and show that one cannot (non-interactively) authenti- 
cate even one bit using (£, £/2)-weak source (this bound is tight as Maurer and 
Wolf [11] showed how to authenticate up to £/2 bits when b > £/2). 


USING SEVERAL WEAK SOURCES. Instead, we will assume that we have several 
weak sources, each one independent from all the other weak sources. Specifically, 
we will try to extract nearly ideal randomness from two weak sources. 

The question of extracting randomness from two or more independent ran- 
dom sources originated in the works of Sántha and Vazirani [20, 25, 27]. Chor 
and Goldreich [4] were the first to consider general weak sources of equal block 
length; let us say that the sources X and Y are (£x, bx )-weak and (fy, by )-weak, 
and for simplicity we also assume fx = ly = £. It is known [7] that in this setting 
it is possible to extract nearly all the entropy (i.e., nearly bx + by — 2log(1/e) 
bits) from the two sources, where € is the required distance from the uniform 
distribution. However, best known explicit constructions achieve much weaker 
parameters. Specifically, for one bit extractors Chor and Goldreich showed that 
the inner product function works provided bx + by > £+ 2log(1/e). The 
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best multi-bit two-source extractor ([4], thm 14) is capable of extracting almost 
5(b x + by —£—2log(1/e)) random bits, but [4] only show how to efficiently im- 
plement this extractor when bx + by > 1.752. It was also known how to extract 
a non-constant bit (or a few bits) from two sources when bx = £(§ +0(1)), by = 
O(log(2)) [9,1], and this method can also be used to extract unbiased bits from 
such sources. In a setting of more than two weak sources, Barak et al. [3] re- 
cently show how to extract randomness from (1/6)? independent (£, 5é)-weak 
sources (for any 6 > 0), but this improvement does not apply to the case of two 
sources considered in this work. 

Motivated by cryptographic applications, Dodis and Oliveira [7] recently con- 
sidered the problem of extracting random bits from two independent sources 
which are (essentially) independent from one of the two sources (called the 
“seed”). They termed this two-source extractor a strong blender, since it is a 
common generalization of a two-source extractor (which they simply termed 
“blender’’) and a strong extractor [16], where the seed source is assumed to be 
public and truly uniform (and the objective of the latter is to make the seed as 
short as possible). They showed the existence of such strong blenders, whenever 
bx, by > log £+2log(1/e), which are capable of extracting (bx —2log(1/e)) bits 
from X which are independent from the (£, by )-weak seed Y. On a constructive 
side, they showed that the constructions from [4] for two-source extractors (1.e., 
““blenders’’) and the related work of Vazirani [27] for the so-called SV sources [20] 
do extend to give strong blenders. The best such construction in [7] (based on a 
similar construction from [4]) is capable of extracting £(b x +by — €—2log(1/e)) 
bits independent of one of the sources, but this is again efficient only when 
bx + by > 1.752. 

To summarize, while randomness extraction from two independent sources 
is possible, the known constructions and parameters seem far from optimal. 
In contrast, there are many efficient constructions of one-source randomness 
extractors (with truly random seeds) with nearly optimal parameters (see [10, 17, 
21] and the references therein). Our work gives an improvement in the number of 
extracted bits, for the case bx +by > £, allowing for the first time the extraction 
of more than half the total randomness that is present in the sources. 


OUR RESULTS. We give a simple two-source extractor capable of extracting 
max(bx, by) + (bx + by — £—4log(1/e)) bits whenever the sum of min-entropies 
is slightly greater than £: bx + by > £+ 92(polylog(€/e)). Our construction is 
based on two observations. First, a strong blender with good parameters can give 
a two-source extractor (1.e., a “regular blender”) with even better parameters, 
when combined with a seeded extractor (with good parameters). Specifically, if 
a strong blender extracts k > (2(polylog(#/¢)) nearly random bits Z which are 
independent from the weak seed Y, we can now apply a seeded extractor to Y, 
using the just extracted Z as the seed (since Z is “long enough” to be used 
with state-of-the-art extractors such as [17]). This allows us to extract a total of 
(k+by —2 log(1/e)) bits, which could be very high. While this obvious observation 
would already improve the state-of-the-art in the problem of two-source extrac- 
tion, even when combined with the known strong blender constructions from 
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[7], our second observation is a much simpler (and better) construction of strong 
blenders. Specifically, we show how to extract (bx +by —£—2 log(1/e)) bits which 
are independent from one of the sources. Thus our strong blender construction: 
(1) improves by a factor of two the best previously published construction of [7]; 
(2) is very simple and efficient without any limitations on bx and by (as long as 
bx +by > £+2log(1/e)); and (3) uses only trivial properties from linear algebra 
in its analysis. By choosing the initial “seed” source to be the one containing 
more min-entropy, we get our final two-source extractor. 


OUR STRONG BLENDER. In fact, we give the following general technique for 
constructing our strong blender. Assume we wish to extract k nearly random 
bits from X using Y as a seed. Let Aj,..., Ax be some £x £ matrices over 
GF[2], specified later. View X and Y as -bit vectors, and output bits ((A;X) - 
Y,..., (Ak X): Y), where - denotes the inner product modulo 2, and A;X is 
the matrix-vector multiplication over GF[2]. Of course, the main question is 
which condition on matrices A;...A, would allow us to maximize the value of 
k. As we show, all we need is to have every non-empty sum Ag = J ieg Ai 
(where S # @)of these matrices to have “high” rank. In particular, by using a 
known linear algebra construction [14, 13, 19]) of such matrices, we can ensure 
that all the needed matrices Ag will have full rank, which will in turn give us 
the desired value k = (bx +by — £—2log(1/e)). However, we also notice that by 
using several other simple, but sub-optimal choices of matrices A; ...A,, we will 
obtain two of the three constructions from [7] as special cases, as well as several 
other constructions which appeared in the literature for related problems (..e., 
[27]). Thus, our methodology elegantly unifies several of the previous approaches 
into one elementary framework. 


2 Preliminaries 


2.1 Basic Notation 


We mostly employ standard notation. The symbol log is reserved for the base 2 
logarithm. For a positive integer t, U, denotes a random variable that is uniform 
over {0,1} and independent of all other random variables under consideration. 
We also write |t] = {1,2,...¢}. For two random variables A, B taking values in 
the finite set A, their statistical distance is |A — Bl|, = $ rac, | Pr[A =a] — 
Pr[B = all, and the min-entropy of A is Hə (A) = minaca ( — log(Pr[A = a])). 
For a random variable A taking values in {0,1}, the bias of A is its statistical 
distance from U, ||A — U;||,. The Lo norm of a vector v € R is |lv||, = 


>~;-1(%)?. For two strings s,t, their concatenation is denoted (s , t). For a 
vector a € {0,1}* and 1 <i < t, we say that i € a ifa; = 1. Also recall that 
an (£,b)~source denotes some random variable X over {0,1}* with min-entropy 


Hoo (X) >b. 


2.2 Strong Blenders and Extractors 


We start by defining the notion of a randomness extractor [16]. 
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Definition 1 ([16]). Let b > 0, € > 0. A (b,e)-extractor Ext : {0,1} x 
{0,1}* — {0,1} is an efficient function such that for all (£,b)-sources Y, we 
have 

| Ext(Y, Us) — Um ll, <€ 


Clearly, one of the main objectives in the area of building explicit extractors 
is to minimize the seed length k while still extracting nearly all the randomness 
from the source. Many nearly optimal constructions exist by now (see [15, 23, 10, 
17, 22, 21] and the references therein). While none of them clearly beats the rest 
in all parameters, it is known (e.g., [17]) how to extract m = k + b — 2log(1/e) 
nearly random bits (which is optimal) using a seed of length k = O(polylog(@/¢)). 

In this work, however, we are interested in (strong) randomness extraction 
from two weak sources. Correspondingly, we define the notion of two-source 
extractors (aka “blenders”) and strong blenders. For simplicity, we assume both 
sources have the same length £ throughout the paper. 


Definition 2. A (bx, by,¢€)-strong blender (SB) is an efficient function BLE : 
{0, 1}£x {0,1}£ > {0,1}* such that for all (£,bx)-weak sources X and all (£, by )- 
weak sources Y, we have 


| (Y , Bre(x,Y)) - (Y , Us) ||, <€ 


A (bx, by, €)-two-source extractor is an efficient function TWo-EXT : {0,1} x 
{0,1} — {0,1}™ such that for all (£,bx)-weak sources X and all (£, by )-weak 
sources Y, we have 

|Two-ExT(X,Y) — Umll, < € 


We now observe the following relation among these three primitives which 
follows immediately from the triangle inequality: namely, given X, Y, the output 
of a strong blender BLE(X, Y) can be used as a random seed for an extractor 
EXT, to extract the randomness from Y. 


Lemma 1. Let BLE: {0,1} x {0,1}* — {0,1}* bea (bx, by, €)-strong blender 
and Ext : {0,1}* x {0,1}* — {0,1}™ bea (by,¢)-extractor. Then the following 
function Two-ExT : {0,1} x {0,1} — {0,1}™ is a (bx, by, 2€)-two-source 
extractor: 


Two-ExT(X,Y) = Ext( Y, BLE(X, Y ) ) 


Our final two-source extractor will follow by applying the Lemma above 
to the strong blender which we construct in the next section (using any good 
seeded extractor, such as the one in [17]). Thus, from now on we concentrate on 
improved constructions of strong blenders. 


3 Efficient Strong Blender Constructions 


In this section we show a construction of an efficient strong blender. In 3.1 we 
give a general technique for constructing a strong blender, that requires a set 
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of £ x £ matrices over GF[2] with the property that the sum of every subset of 
matrices has high rank. In section 3.2 we give an explicit set of matrices (that 
appeared in [14, 13, 19]), with the property that the sum of any subset of matrices 
has full rank. This gives the best possible result for our technique. In the same 
section we also review some previous constructions for two-source randomness 
extraction as special cases of our technique. 


3.1 General Technique for Constructing Strong Blenders 


We now show how to extract many random bits from two weak sources X, Y of 
the same length £, such that the bits are random even if one sees Y. 
Let Aj,..., Ax be £x £ matrices over GF [2], such that for every nonempty subset 


S C [k], the rank of As = ee A; is at least £ — r, for O < r < £ (we will want 
to have high rank and keep r as small as possible). 
The proposed strong blender is 
BLE4 : {0,1}* x {0,1} — {0,1}* ' 
(2:9) m (12) -Yy,..., (Ape) -y) (1) 


where - is the inner product mod 2 and A;x is a matrix-vector multiplication 
over GF[2]. 


Theorem 1. The function BLE, is a (bx, by,€)-SB 
with log 4 = 2x thy tee (ftrt) | that is 


|(Y , Buea(X,¥)) = (Y , Ua), <2 


Our main tool for proving theorem | is the Parity lemma (also known as the 
XOR lemma, see [8, 27]), which relates the bias of a k-bit random variable T 
with the sum of squared biases of every subset of the bits in 7. 


Lemma 2 (Parity lemma [8]). For any k-bit random variable T, |T — Ux|\, 
is upper bounded by 
> a Ol 


0k Aa€{0,1}* 


where T -a is the inner product (mod 2) of T and a. 


We also use a relation between the rank of a matrix L and the min-entropy 
of LX. 


Lemma 3. Let L be an £x £ matrix over GF[2]. If the rank of L is 2—r, then 
for any random variable X over {0,1}, Ho. (LX) > Hao (X)— Tr. 


Proof. Note that for any z € {0,1}*, there are at most 2” different values 
x € {0,1}* such that Læ = z. Thus, maxzeroije Pr[LX = 2} < 2. 
maXre{0,1} Pr[X = z}. 


340 Yevgeniy Dodis et al. 


Proof. [of Theorem 1] Following [4, 7], it is sufficient to consider the case when 
Y is uniformly distributed on some set Sy, |Sy| = 2Y, and X is uniformly 
distributed on some set Sx, |Sx| = 2°*. 


I(Y , BLea(X,Y))- (Y , Uk) = XO ||BLEa(X,y) — Ukl 





Fr [Syl yeSy 
1 
(by parity lemma) < >D Sy] 2. (IBLEA(X, y) -a — Uill)? 
ye Sy ok aE {0,1}* 
(by concavity) < 5 =D |BLEA(X,y)-a — Villy) 
0k 4a€{0,1}* [Sy] Y| yESy 


We now proceed to upper bound each of the (above) bracketed terms by 
2~(bx+by+2—£-r). the final result then follows from simple addition. Fixing a 
value of a = ayaz...a% € {0,1}*\{0*}, and identifying a with the set of all 
i € [k] for which a; = 1, we have 


BLEg(X,y)-a = S_ BuEa(X,y); = S (AX) -y = (so) "Y 


i€a i€a i€a 
where all sums are taken modulo 2. 
Now define A, © J ica Ai (mod 2), and recall that the rank of Ag is at 


least  — r. As a proof tool, we introduce the 2 x 2 matrix M whose rows 
and columns are labeled by the elements of {0, 1}4, and whose (x, y)th entry is 


Mii = (—1)(4e2)-9 (for z,y € {0, 1}*). We also let M be the xth row of M. 
The following two properties are the key to what follows. 


1. Every row of M is equal to at most 2" other rows of M. Indeed, if x,x’ € 
{0,1}¢ satisfy M+ = Mx, then it must be that Aaz = Aar’ (mod 2), i.e. 
x +z’ (mod 2) is in the kernel of Aa. Since the rank of A, is at least £ — r, 
the kernel of A, has at most 2” elements. 

2. Rows ofM that are distinct are orthogonal (ue in the Euclidean inner product 


sense). For if x Æ x’ € {0,1}* are such that M, x Mv, then z & A alz + 
r') # 0° (mod 2), and the R-valued Euclidean inner product of M, and 

ae 

My = = (M Ma)! (Mz! )= = deye{o,1}4(— 1)” PASOS is 0. 


Now let D € R? be the vector defined by De 5x] Dres M,. It is 


not hard to see that the yth coordinate D, of D is equal to $(Przesx [Mz,y = 
1] — Prees, [Mz = —1]), so that |D,| = ||BLE4(X,y)-a — U,||,. The quantity 
we wish to bound is thus 


5 D> (Buea(X,y)-a = Uila)? = a SDP +p 


yESY ye Sy 
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—|\2 n> 
and [>], equals the Euclidean inner product DT D over R 


—> 112 1 nak = 
>I, = Sep, Do, Ma) (Me) 


But properties 1. and 2. above show that for any x € Sx 


S M Me) = V (MMe) + Y (Mz) (Mz) 


r'ESX sM =Ma zM #Ma 














2 
Hence [B| < 2"+£/4|Sx|, and plugging this bound into (3.1) yields 


or+é 
5 (|[Buba(X,y)-@ S < <a = erte) 
yESY 4| Sx ||Sy| 


1 
[Sy | 


which, as noted above, implies the theorem. 


We emphasize that the above technique is entirely general: any set of matrices 
Aj,...,Ax such that the sum of any non-empty subset of matrices has rank of at 
least 2—r, can be used to extract k < by + by + 2 — l — r bits from €-bit weak 
sources X, Y with respective min-entropies bx, by. 

The following two cases are of special interest (of course, they apply only 
when r is “small enough”; that is, when r + k < by + by + 2 — £): 


1. If the min-entropies bx + by sum up to £+ polylog(£), we can extract poly- 
logarithmic (in £) number of bits with bias 2~(Polylog(®) 

2. If the min-entropies bx + by sum up to (1+ c)é, for any constant 0 < e < 1, 
we can extract linear (in £) number of bits with exponentially small bias. 


3.2 Explicit Strong Blender Instantiations 


Theorem 1 subsumes constructions introduced (sometimes implicitly) in pre- 
vious works. These constructions are presented below, followed by our more 
randomness-efficient construction. 


IDENTITY MATRIX. This is the simplest case when k = 1 and A; is the identity 


matrix. This gives r = 0 and implies that the inner product function is a strong 


blender with bias € = aria laa reproving the result from [7] (adapting the 


result from [4]). 


CYCLIC SHIFT MATRICES. Vazirani [27] used cyclic shift matrices, to extract 
randomness from SV sources. When £ is a prime with 2 as a primitive root, the 
sum of any subset of matrices has rank at least 2—1 (that is r = 1, in our notation 
above). Formally, let A; be a linear transformation matrix corresponding to a 
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cyclic shift by 2 — 1 bits; That is, the jth row of A; has a 1 in the 7 — i+ 1 
(mod £) column and zero elsewhere. The required property of these matrices is 
given below. 


Lemma 4 ([27]). Let £ be a prime with 2 a primitive root modulo £ (i.e. 2 is a 
generator of Zł). Let œ € {0,1}* \ {0°,1°} be a vector (which is not the all 0’s 
or the all 1’s vector). Let A be an £ x £ matrix over GF(2), such that the rows 
of A are the £ right-cyclic-shifts of w. Then rank(A) > @—1. 


By Theorem 1, a corresponding strong blender can extract k bits with bias 
bx tby 1—f—-—k 
ain: i , or, equivalently, extract k = (bx +by +1 — £ — 2log(1/£}) bits 
with bias €. 


(NON-CYCLIC) RIGHT SHIFT MATRICES. Let Aj,...,Axz be linear transforma- 
tion matrices such that A; corresponds to the right shift by 7 — 1 bits. For any 
non-empty subset S C [k], it is easy to see that the rank of As = ieg A; is 
at least £ — k + 1. By theorem 1 one can build a strong blender from those 
matrices that extracts k bits with bias 2- “== 
k = (bx + by — £ — 2log(1/e))/2 bits with bias e€. 


THE MATRICES FROM A GENERAL ERROR-CORRECTING CODE. The construction 
of [7] (adapted from [25]) also pertains to the present framework. Let C be 
an [£,k,d] linear error correcting code (i.e., having dimension k and minimal 
distance d in {0,1}*). Let A1,...,Ax be diagonal matrices, the diagonal of A; 
containing the coordinates of a codeword c; encoding the i-th unit vector in 
{0,1}*. By linearity, the diagonal of As = };eg A; is also a codeword in C, 
which is non-zero when S Æ Ø, and thus has at least d ones, so that rank(Ag) > 
d. Then Theorem 1 applies with r = £— d, and the corresponding strong blender 
l i .  „— (bx +by +d—2t—k) l 

extracts k bits with bias 2 2 , or, equivalently, extract k = (bx + 
by +d — 2 — 2log(1/e)) bits with bias € (notice, sometimes this k may not be 
achievable due to the coding constraints). 


, or, equivalently, extract 


OUR CONSTRUCTION. We now present our new construction of strong blenders 
via Theorem 1. We show that there is a set of matrices A;,...Ag whose non- 
empty sums have full rank, thus achieving the best possible result for using 
the technique in section 3.1. This latter fact follows from [14, 13, 19], and we 
reproduce its simple proof for completeness. We use the isomorphism {0,1} = 
GF[2°). Take any basis x1, 22,...,2¢ of GF[2f]. (There are many such bases for 
G F/2°|, since it is isomorphic to the -dimensional vector space over GF[2]. One 
such basis, which is easy to use, is 1,2, £°,.. .,2*-! where x is any primitive 
element of GF(2°|; that is, an element that is not the root of any polynomial of 
degree < Zover GF[2*|.) The matrices {A;}£_, are defined such that matrix A; 
corresponds to left-multiplication by the basis-element z;: 


A; y E€ GF[2}] = ay 
Now for each non-empty set S C [8], zs = J ieg zi is a non-zero element in 


GF([2‘], and therefore has an inverse xz* in GF[2°]. Let Us be the £ x £ matrix 
over GF[2] that corresponds to left-multiplication by tg" 
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Us: y € GF[2f] > z3ty 


Now, it is easy to see that Us is the matrix-inverse of As = ) jeg Ai, since 
left-multiplication by matrix Us }7,<, A; corresponds to multiplication by the 
field identity element x5" Y ,¢5 T: = 1. 

This immediately implies the desired conclusion: 


Lemma 5. Let A,,...,A¢ be matrices presented above. Then for all® # S C [8], 


the rank of As = dues A is t. 


Plugging the first k of the matrices A,,...,Ag into theorem 1 we deduce 
the existence of an explicit strong blender that can extract k bits with bias 


cena eae or,equivalently,extract k = (by +by +2—@—2log(1/e)) bits. 
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Abstract. The central observation of this paper is that if en random 
edges are added to any n-node connected graph or digraph then the 
resulting graph has diameter O(log n) with high probability. We apply 
this to smoothed analysis of algorithms and property testing. 
Smoothed Analysis: Recognizing strongly connected digraphs is a ba- 
sic computational task in graph theory. Even for graphs with bounded 
out-degree, it is NL-complete. By XORing an arbitrary bounded out- 
degree digraph with a sparse random digraph R ~ Dn,e/n we obtain a 
“smoothed” instance. We show that, with high probability, a log-space 
algorithm will correctly determine if a smoothed instance is strongly 
connected. We also show that if NL Z almost-L then no heuristic can 
recognize similarly perturbed instances of (s, t)-connectivity. 

Property Testing: A digraph 1s called k-linked if for every choice of 2k 
distinct vertices s1,...,8%,t1,...,tx, the graph contains k vertex disjoint 
paths joining s» to tp for r = 1,...,&. Recognizing k-linked digraphs 
is NP-complete for k > 2. We describe a polynomial time algorithm 
for bounded degree digraphs which accepts k-linked graphs with high 
probability, and rejects all graphs which are at least en edges away from 
being k-linked. 





1 Introduction 


The diameter of a graph G is the length of the longest shortest path in G. 
In other words, if d(u,v) is the length of the shortest path from u to v in G, 
then the diameter of G is max, d(u,v). A graph is connected (and a directed 
graph is strongly connected) if it has finite diameter. The central observation 
of this paper is that if en random edges are added to any n-node connected 
graph then the diameter becomes O(log n) with high probability (meaning with 
probability tending to 1 as n — oo and abbreviated whp). This is also true for 
strongly connected directed graphs (digraphs) and for several ways of generating 
the random edges. For ease of exposition, we state this as a theorem only for a 
strongly connected digraph perturbed by adding a random digraph R ~ Dy, jn. 
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Here R ~ D means R is distributed according to distribution D, and D,,, is the 
distribution over n-node digraphs where each arc is included independently with 
probability p. 


Theorem 1. Lete be any positive constant. For any strongly connected n-node 
digraph D, let D = D+ Rwhere R ~ Dy ejn. Then whp the diameter of D is 
at most 100e~" logn. 


Similar results hold for perturbations formed by adding en edges selected at 
random with or without replacement, or by adding a random matching or as- 
signment with en edges, or by picking en vertices and adding a uniformly random 
out-arc to each of them. 

Theorem 1 is related to a class of problems regarding the possible change of 
diameter in a graph where edges are added or deleted, for example, the results of 
Alon, Gryarfas, and Ruszink6 in [2] on the minimum number of edges that must 
be added to a graph to transform it into a graph of diameter at most d. The 
study of these extremal diameter alteration questions was initiated by Chung 
and Garey in [12]. It is also related to the theorem of Bollobas and Chung on 
the diameter of a cycle plus a random matching [10]. 


1.1 Application: Smoothed Analysis 


Recognizing strongly connected digraphs is a basic computational task, and the 
set of strongly connected digraphs is NL-complete [21]. Thus, if NL Z L then 
there is no log-space algorithm which recognizes strongly connected digraphs. 

Perhaps this conclusion of worst-case complexity theory is too pessimistic. 
We will consider the performance of a simple heuristic which runs in randomized 
log-space. We will show that the heuristic succeeds on random instances whp. 
However, the “meaning” of this result depends on the probability space from 
which we draw the random instances. It seems reasonable to assume that any 
real-world digraph will contain some amount of randomness, so it is tempting to 
believe this result shows that in the real-world strong connectivity only requires 
log-space. Unfortunately, this is not valid if we use the “wrong” model for ran- 
domness. For example, the distribution Dn, p is pleasant for analysis, but basic 
statistics like degree sequence seem to differ from real-world graphs [14]. 

We will use a model of randomness that is more flexible. We will start with 
an arbitrary digraph D and perturb it by XORing it with a very sparse random 
graph R ~ Dn,e/n. This produces a random instance which is “less random” 
than Dn,p. The study of worst case instances with small random perturbations 
is called Smoothed Analysis. 

Smoothed Analysis was introduced by Spielman and Teng in [27] and they 
discuss a perturbation model for discrete problems in [28]. They consider per- 
turbing graphs by XORing the adjacency matrix with a random adjacency ma- 
trix, where each edge is flipped with some constant probability. Since the prob- 
ability of an edge flip is constant, the perturbed instances are all dense graphs 
(i.e. a constant fraction of all possible edges appear). Independently, Bohman, 
Frieze and Martin [7] studied the issue of Hamiltonicity in a dense graph when 
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random edges are added, and other graph properties were analyzed in this model 
by Bohman, Frieze, Krivelevich and Martin [8] and Krivelevich and Tetali [23]. 

We will also use an XOR perturbation, but we will make the probability of 
corruption much lower than [28]. Since we will have a linear number of edges 
present, it is appropriate for the perturbation to change about en edges, which 
is the expected number of edges in D, c/n- 


Randomness and Strong Connectivity. Recognizing strongly connected di- 
graphs is closely related to recognizing (s,t)-connectivity in digraphs, which is 
the canonical NL-complete problem. It is possible to recognize connectivity in 
undirected graphs with a randomized log-space algorithm using random walks 
[1].Since the cover time of an arbitrary connected graph is bounded by O(n?) 
[17], a random walk will visit every vertex in polynomial time whp. This ap- 
proach will not work for arbitrary digraphs, however, since there the cover time 
can be exponential. 

The diameter and connectivity of random graphs has been well-studied, see 
for example the books of Bollobas [9] and Janson, Łuczak, and Ruciński [20]. 
Perhaps closest in spirit to our investigation is the paper of Bollobas and Chung 
on the diameter of a Hamilton cycle plus a random matching [10] and the paper 
of Chung and Garey on the diameter of altered graphs [12]. Also, the component 
structure of random digraphs was studied by Karp in [22] and more recently by 
Cooper and Frieze [13]. 


A Heuristic for Recognizing Strong Connectivity. For each ordered pair 
of vertices (s,t) repeat the following procedure N; times: Starting from s, take 
Nə steps in a random walk on the digraph. Here a random walk is the sequence 
of vertices Xo,X1,...,Xz,... visited by a particle which moves as follows: If 
X, =v then X;4, is chosen uniformly at random from the out-neighbors of X+. 
If the random walk with restarts ever reaches t, then the digraph contains an 
(s,t)-path, and we continue to the next pair of vertices. 

If Ny and No are large enough, this algorithm is correct whp. For example, if 
there is a pathfrom s to t and N; = n?", No = n we will discover the path whp. 
The values of N,; and N> can be significantly improved for smoothed random 
instances 

The main theorem of this section is that when N; and Nz are suitable poly- 
nomials in n, this heuristic, which we will call Algorithm A, is successful on per- 
turbations of bounded out-degree instances whp. To prove this, we first show 
it is successful when the initial instance is a strongly connected digraph and 
the perturbation only adds edges. Then we extend this to show success when 
the initial instance is not necessarily strongly connected and the perturbation 
only adds edges. After this, it is simple to translate our results to the original 
perturbation model where edges are added and removed, since we can gener- 
ate the perturbation in 2 rounds, by first deleting each existing edge with some 
probability, and then adding random edges to the resulting digraph. 
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We write G1 @ G2 to mean the XOR of digraphs G; and G2, (which is to say 
e € G; © Go if and only if e € Gi and e ¢ Go or vice versa.) 


Theorem 2. Let e be any positive constant. For any n-node digraph D with 
maximumout-degree A, let D = D@Rwhere R ~ Dn ejn. Then whp Algorithm 


-1 _ 
A is correct on D when N; = n41€ 8004) and Na = Bye“ logn . 


Here A; and Bı are absolute constants. We find that A; = 400 and A» = 200 
suffice, but we do not attempt to optimize these values. 

If a strongly connected digraph has bounded out-degree and has diameter 
O(log n) then a random walk of length O(log n) has a 1/ poly(n) chance of going 
froms to t, and Algorithm A will succeed whp using values of N; and N2 that 
can be realized in log-space. Unfortunately, even though our initial instances have 
bounded out-degree and (as indicated by Theorem 1) our perturbed instances 
have logarithmic diameter, the perturbation increases the maximum degree to 
N(log n/loglogn), so we must work a little harder to show that the random 
walk has a non-negligible probability of witnessing the path from s to t. (As an 
additional reward for this work, we find that Algorithm A can be derandomized 
by checking all paths from s of length O(e~! logn) and still only use log-space.) 

The analysis of Algorithm A is further complicated by the possibility a D 
which is not strongly connected combining with R to produce a smoothed in- 
stance which is strongly connected. We handle this situation by arguing that for 
the smoothed instance to become strongly connected there cannot be too many 
small strong components of D, and then the large components become strongly 
connected with short paths that a random walk is not too unlikely to travel, and 
finally, the small components, if they are connected to the large component, are 
“close” to it. 


Why Study Instances with Bounded Out-Degree? It would be nice to 
extend our results to hold for perturbed copies of any digraph, instead of only 
digraphs with bounded out-degree. However, such a result is not possible for our 
heuristic. We demonstrate the necessity of our assumption that D has bounded 
out-degree by constructing a family of instances with growing degree on which 
Algorithm Adoes not succeed whp. 


Theorem 3. Let f(n) — oo, let e be a positive constant less than 1, and let 
R~Dnie/n. Then for every sufficiently large n, there exists an n-node digraph 
D with maximum out-degree A = f(n) such that for D ® R the probability 
Algorithm A fails exceeds 1 — e~€ — o(1). 


The proof of Theorem 3 is omitted for lack of space. 


Strong Connectivity Versus (s,t)-Connectivity. Strong connectivity is an 
NL-complete problem, but (s,¢)-connectivity is “the” NL-complete problem. In 
Sipser’s undergraduate text [26], the completeness of (s, ¢)-connectivity is proved 
in detail, while the completeness of strong connectivity is left as an exercise. 
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In light of this, it is natural to investigate the efficacy of heuristics on 
smoothed instances of (s,t)-~connectivity. Here we find that there are instances 
on which Algorithm A fails whp. What is more, no log-space heuristic exists, 
provided a conjecture of complexity theory holds. 


Theorem 4. If NL Z almost-L then no log-space heuristic succeeds whp on 
smoothed instances of bounded out-degree (s,t)-connectivity. 


The proof consists of building a machine which simulates any nondeterministic 
log-space machine using the log-space heuristic for (s, t)-connectivity, were such 
a heuristic to exist. The proof is omitted for lack of space. 


Smoothed Model Versus Semi-random Model. The semi-random model 
was introduced by Santha and Vazirani in [25]. In this model an adversary adap- 
tively chooses a sequence of bits and each is corrupted independently with prob- 
ability 6. They present this as a model for real-world random bits, such as the 
output of a Geiger counter or noisy diode, and consider the possibility of using 
such random bits in computation on worst-case instances. Blum and Spencer 
considered the performance of a graph coloring heuristic on random and semi- 
random instances in [6]. Subsequent work has uncovered an interesting difference 
between the random and semi-random instances in graph coloring. The work 
of Alon and Kahale [3] developed a heuristic which succeeds whp on random 
instances with constant expected degree, while work by Feige and Kilian [16] 
showed no heuristic can succeed on semi-random instances with expected degree 
(1 — €)logn (they also developed a heuristic for semi-random instances with 
expected degree (1 + €) logn). 

In the original semi-random model of Santha and Vazirani, an instance is 
formed by an adaptive adversary, who looks at all the bits generated so far, 
asks for a particular value for the next bit, and gets the opposite of what was 
asked for with probability 6. Several modifications are proposed in Blum and 
Spencer [6] and also in Feige and Krauthgamer [15]. However, all these variations 
maintain the adaptive aspect of the adversary’s strategy, which at low density 
allows too much power; if the error probability p = (1 — €)logn/n then there 
will be roughly n* isolated vertices in Dn,p and the adversary will be able to 
encode a polynomial sized instance containing no randomness. Since we wish to 
consider extremely sparse perturbations, where the error probability p = €/n, 
we cannot allow an adversary as powerful as in the semi-random model. The 
XOR perturbation considered in this paper is equivalent to a natural weakening 
of the semi-random model: making the adversary oblivious. 


1.2 Application: Property Testing 


Property testing provides an alternative weakening of worst-case decision prob- 
lems. It was formalized by Goldreich, Goldwasser, and Ron in [18]. The goal 
in property testing is to design an algorithm which decides whether a instance 
has a property or differs significantly from all instances which have that prop- 
erty. For example, a property tester for strong connectivity in bounded degree 
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digraphs should accept all strongly connected instances and reject all instances 
that are en edges away from being strongly connected. Note that Algorithm A 
(which is designed to work on smoothed random instances) can be converted 
into a property tester: given an instance D and a gap parameter €, we can ran- 
domly perturb D ourselves by adding 5n random edges and then run Algorithm 
A on the perturbed version. This does not yield anything impressive for testing 
strong connectivity, since the undirected connectivity testing results of Goldreich 
and Ron in [19] can be applied to the directed case to produce a constant time 
tester. However, our perturbation approach also yields a property tester for a 
more difficult connectivity problem, k-linked. 

A digraph is called k-linked if for every choice of 2k distinct vertices s1,..., Sk, 
t1,.-.,¢,, the graph contains k vertex disjoint paths joining sı to t1, ..., Sk to tk. 
Recognizing k-linked digraphs is NP-complete for k > 2. In the bounded degree 
property testing version of k-linked, we are given a constant e and a digraph D 
with maximum out-degree A and our goal is to accept if D is k-linked and reject 
if D is more than en arcs away from being k-linked, and we can do whatever if 
D is not k-linked, but is close to being so. 


A Heuristic for Testing k-Linkedness. Given D and e, we perturb D by 
flipping a coin for each node, and with probability €/2 adding a out-arc leading 
to a node chosen uniformly at random. Call this perturbed instance D. Then, 
for each choice of 2k distinct vertices, repeat the following procedure Nj, times: 
For i =1,..., starting at s; take Ne steps in a random walk on the graph. If 
all k random walks ever reach the correct k terminals via vertex disjoint paths, 
we continue to the next choice of 2k vertices. Otherwise reject. 
Here k is assumed to be fixed, independent of the input. 


Theorem 5. Let € be any positive constant. For any k-linked n-node graph D 
with maximum degree Á, the algorithm above accepts whp. 


A few comments regarding the difference between this theorem and Theorems 
1 and 2. Since we perturbed D by adding a 1-out, the maximum out-degree 
of D is bounded by A+ 1, so we are not faced with one of the additional 
challenges in Theorem 2 (which is addressed by Lemma 1); if & vertex-disjoint 
logarithmic length paths appear then we can immediately conclude that whp 
they will be discovered after a polynomial number of samples. What is more, 
the fact that such paths exist whp follows from a calculation analogous to the 
proof of Theorem 1, that explores disjoint neighborhoods around all 2k terminals 
simultaneously. Also, the analog of the most difficult part of Theorem 2, showing 
that Algorithm A is correct in the case where a disconnected D leads to a strongly 
connected D, is no longer necessary. In the property testing setting, we are not 
required to correctly recognize instances that lead to this situation. It seems as 
though it might be possible to carry out this most difficult part and obtain a 
heuristic for k-linked that works on smoothed instances, but the details remain 
elusive. 
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2 Proof of Theorem 1 


We now show that if D is strongly connected then the diameter of D = D + R 
is O(logn) whp. 

Instead of adding a random digraph R with probability p = e/n, we will 
add two random digraphs R’ and R” with arcs appearing independently with 
probability p’ and p” respectively. We will take p’ = et = (1—o(1))e/n and 
p” = n-7/®, The probability of an edge appearing in the union of R’ and R” 
is p' + p” — p'p" =e/n. So adding R’ and R” to D yields a digraph identically 
distributed with D. 

We will show that whp D contains short paths of a special form, alternating 
between an appropriate number of edges from D and a random edge of R’. This 
is similar to the approach of Bollobas and Chung [10]. 

Let So = To = 9, and let Så (resp. Tj) be the set of vertices s§ (resp. t) for 
which there is a path of length at most 32e-!logn + 1 from s to sh (resp. from 
th to t) in D. 

Now, for 2 > 1, we identify a set S; of neighbors of S;_, and then a set S; of 
neighbors of S;, and do the same for T; and Ty. 

Let S; (resp. T;) be the set of s” ¢ Ha Sj U S; (resp. t” g Ua T; UT;) for 
which the arc (s’, s”) (resp. (t’”,t’)) isin R’ for some s’ € Sj_,. 

Let S; (resp. Tj) be the set of vertices s; (resp. t;) for which there is a path 
in D of length at most 

dabe tpi 


from some s; E S; to s; (resp. from t; to some t; € T;). Let 














£ = [logan]. 
e £ 
Let Zs = | U (S; U S;)| and Zr = | U (T; UT)|. We will now show that 
j=0 j=0 
Pr[Zs < n?” or Zr < n?/3] = o(n-?). (1) 


We will argue this assertion only for Zs. The part regarding Zr is proved anal- 
ogously. 

Let U; = U (Si U S!). For i < £, suppose that |U;| < n?/. Let s” be some 

j=0 

vertex not in U;. Then the probability that there is an arc in R’ connecting Sj_, 
to s” is 1— (1 = p’)!S-al = (1 — o(1))e E, 

We use the following Chernoff bounds from [20, Theorem 2.1] on the Binomial 
random variable B(n, p) 


2 
BBG p Sapo ses l-am. (2) 


a E E T [5 | (3) 
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Since the events {s” € S;} are all independent, we let Z = |S;| and see that 
Z is distributed as B(n — o(n), (1 — 0(1))e|S{_,|/n). So from (3) we have 


a sil < $ls Usp sw, S alae a, 





Now, for |.$;| < n?/’, we will show that |.$/| is likely to be at least 4e~1),S;]. 
Note that given its size s, the set S; is a random s-subset of the vertices not in 
Ui. Since |U;| < n?/3, there are n — o(n) vertices to choose from. And since the 
out-degree of D is bounded by A, there are at most A”? vertices within distance 
2d of a given vertex. We will call a randomly selected vertex a failure if it is 
at distance less than 2d from a previously selected vertex. Then the probability 
of failure of a particular vertex, conditioned on the success or failure of any 
other vertices is at most (1 + 0(1))|S;|A2¢4/n. So the number of failures Np is 
stochastically dominated by B ({.5;|,q), where q = (1+0(1))|S;{A24n~1. Since we 
are interested in a wide range of values of |5;|, we bound the same probability for 
large and small values of |S;|. If |S;| > n5/1* we use (2) to obtain the following: 


Pr | Ny > Silga +n! 





Ui] < Taai si 


nels P ni/3 
< Sipe eee ee eee Pewee 
<exp{ CTE } < exp 4 red 


which relies on the fact that |,9;| < n?/3 and hence |S;\q < (1 + o(1))A2¢n1/3. 
If |S;| < n5/12, then we use the union bound to show 


|S; | |S;|A24 18 io ae of 


In either case, we have that the failure set has size o(|S;|)} with probability 
1 — O(n-~3). We now obtain a lower bound on the size of S} by ignoring all the 
failure vertices. Note that the neighborhoods of size d of the non-failure vertices 
are disjoint, and since D is strongly connected, there are at least d — 1 vertices 
within distance d of every vertex. So 


[Sil 2 (1 — o(1))5e~"|S;| 


with probability 1 — O(n~*). Combining this with our bound on the probability 
that |S;| < Ze|S}_,| gives 


= isi S 21Si_a| | [Ui] < 7/9, |S7_y|] < eo! + O(n). 
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Again using the fact that D is strongly connected, we have |54| > 3267} logn, 
so we have 


Pr [0a] < nS] < Pr 3: [Ui] <n2/ and [5%] < 2154.1] 


£ 
< YO Pr [isi < 2181 


t=1 


|U| < n?/3 S] > 327! log | = O(n? logn). 





From the above we have that U = oe S; US; and W = Via T; UT; have 
size at least n?/3 with probability 1 — O(n7). If U, W intersect, then we know 
there is a path of length at most 2(d + 1). Otherwise, we expose the edges of 
R” (which appear independently with probability p” = n—7/*) and find that 


4/3 1/6 


Pr[R"NUxW=0)<(1—p”)” <e™' =0o(n-?). 


Since there are only n(n — 1) choices for s,t, the theorem follows. iJ 


3 Sketch of Proof of Theorem 2 


3.1 When D Is Strongly Connected 


By Theorem 1 we know that the diameter of D is O(log n) whp. Unfortunately 
we cannot yet conclude that Algorithm A is successful whp. We must still 
argue that the probability of a random walk traversing the short path is not too 
small. In the original graph, D, having a diameter of O(logn) would imply an 
efficient algorithm, because the maximum out-degree is bounded. Our random 
perturbation has created some vertices with out-degree (log n/ log log n), so we 
will have to work a little more. 


Lemma 1. Let D = D+ R, where D is an arbitrary digraph with maximum 
out-degree A and R ~ Dn ejn- Then whp D contains no path P of length £ < 


lo = 100e~" logn with T],-p degp(z) > n1007? log(10.4) 


The proof of Lemma 1 is omitted due to lack of space. _ 
The correctness of Algorithm A in the case when D is strongly connected 
now follows from that fact that the probability a random walk follows a path P 


from s to tis precisely (JJ, ep degt(x))~”. 


3.2 When D Not Strongly Connected 


The previous section shows that Algorithm A is correct whp for strongly con- 
nected digraphs D. To prove that Algorithm A is correct whp when D is not 
strongly connected, we must do some more work. 

Outline of approach 
Consider the strong components of D. If there are many components of size 
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less than se! logn, then we show that whp one of them will be incident to 
no arcs of R and so D will not be strongly connected and Algorithm A will 
be correct. In the case where D consists mostly of larger strong components, 
we expose the random arcs R in two rounds. We argue that whp the strong 
components of D merge into a unique giant strong component containing at 
least n — n16/17 vertices after the first round. Then we invoke Lemma 1 from 
the previous section to show that the random arcs from the second round gives 
the giant a low diameter. Then we deal with the vertices that belong to small 
strong components after the first round of random arcs have been added. These 
vertices might be connected to the giant in both directions and they might not, 
and there is not necessarily a sharp threshold for strong connectivity. However, 
we show that for some constants A; whp no such vertex is connected in either 
direction to the giant only by paths of length more than A,e~!logn i.e. such a 
vertex is close to the giant in some direction, or cannot be reached at all in this 
direction. Finally, by Lemma 1 we know that all the paths of length at most 
A,e~logn have a non-negligible probability of being traversed by Algorithm A 
(take the bound in Lemma 1 and raise it to the power A;/100). So we conclude 
that whp the graph is not strongly connected, in which case Algorithm A is 
correct, or the graph is strongly connected in such a way that Algorithm A is 
still correct. 
The calculations required for this plan are omitted due to lack of space. 


4 Proof of Theorem 5 


We will show that if D is k-linked then whp D contains disjoint paths of length 
at most 100ke~! logn which witness its k-linkedness. 

For a particular choice of s1,...,5%,t1,..-,tk, let Q1,Q2,...,Qx be vertex 
disjoint paths in D such that Q, goes from sp to tpr. 

We order the paths from longest to shortest and define £ so that for r < £ 
each Q, has length at least 100ke~+ log n. If £ = 0 then there is nothing to prove, 
so suppose £ > 1. 

We use the same type of argument as in the proof of Theorem 1 to show 
the existence of short paths between s, and t,, but we work will all r < £ 
simultaneously to ensure that the paths we find are vertex disjoint. To this end, 
we define a sequence of collections of sets S;, and T;,, fori > 0 andl <r < 2, 
(and define Sir = Q; for i > 0 and r > £). Let So» be the first 32ke~'logn 
vertices of Q, and To,- be the last 32ke—! logn vertices of Qr, forl <r <£. 

We will call a node useful if it is not within d = 5e~! of any node which we 
have previously placed in any S or T set. 

To define S;,,, we check, for each node s’ in Si—1,r, if s” has a random out-arce 
between it and a useful node s”. If it does, we add s” and all nodes reachable 
from s” in d steps to Sir. We continue this until |S; | exceeds 2|S;.1,,| (and 
declare failure if it does not). 

T; r is defined analogously, but the paths lead towards t, instead of away. For 
a node t’ in T;_1,,, we look for useful nodes t” where the random arc is directed 
from t” to t’, and put 2|7;_1,,-| of them in Tir- 
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We wish to argue that we can form sets Si, and T; r of the appropriate sizes 
while |Sir| = |Ti,-| < ?/%. Since every time we discover a useful neighbor we 
add d = 5e—! nodes, we only see failure if for some i and r, we find that Si, r has 
less than Žeļ|S; r| useful neighbors. But at any step, there are at most 2kn?/3 
nodes which have been placed in any S or T, and (since the degree is bounded 
by A) this means there must be at least n — A?¢2kn?/3 useful nodes. So each node 
in Sir has a random out-arc to useful neighbor independently with probability 
at least (1 — o(1))e, and using the Chernoff bound (3), we have 


Pr |The number of useful neighbors of Sir < =|Sirl < eTel Sir]. 


Also, each useful node has a random out-arc to a node in T;,, independently 


with probability elTirl. so by (3) we have 


Tt 


Pr |The number of useful neighbors of Tir < -|T| < e7 Tirl/9, 


Since |So,r| = |To,r| = 32ke™! logn, a union bound shows that the probability 
of failure is less than 2kn~** logn. 

To conclude, we note that when |S;,-| = |Ti,,| > 5n?/%, the probability that 
no node of S; has a random out-arc to a node of T; is at most 


€ zn? € 
(1 = sn) < exp rue = alna y: 


So the probability that there are not vertex disjoint paths of length less than 
100e7 1k logn from s; to t; is at most o(n~?*). Thus by the union bound, we see 
that all choices of 2k nodes have short edge disjoint paths linking them whp. 
Since the random 1-out perturbation preserves the boundedness of out-degree, 
random walks will discover these paths with non-negligible probability. C 
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Abstract. Let G(n,c/n) and G,(n) be an n-node sparse random and a 
sparse random r-regular graph, respectively, and let Z(n,c) and Z(n,r) 
be the sizes of the largest independent set in G(n,c/n) and G,(n). The 
asymptotic value of Z(n,c)/n as n — oo, can be computed using the 
Karp-Sipser algorithm when c < e. For random cubic graphs, r = 3, it is 
only known that .432 < lim inf, Z(n,3)/n < limsup, Z(n,3)/n < .4591 
with high probability (w.h.p.) as n — oo, as shown in [FS94] and [Bol81], 
respectively. 

In this paper we assume in addition that the nodes of the graph are 
equipped with non-negative weights, independently generated accord- 
ing to some common distribution, and we consider instead the maxi- 
mum weight of an independent set. Surprisingly, we discover that for 
certain weight distributions, the limit lim, Z(n,c)/n can be computed 
exactly even when c > e, and lim, Z(n,r)/n can be computed exactly 
for some r > 2. For example, when the weights are exponentially dis- 
tributed with parameter 1, lim, T(n, 2e)/n œ% .5517 in G(n,c/n), and 
limn Z(n,3)/n œ% .6077 in G3(n). Our results are established using the 
recently developed local weak convergence method further reduced to a 
certain local optimality property exhibited by the models we consider. 
We extend our results to maximum weight matchings in G(n, c/n) and 


G,(n). 





1 Introduction 


Two models of random graphs considered in this paper are a sparse random 
graph G(n, c/n) and a sparse random regular graph G,(n). The first is a graph on 
n nodes {0,1,...,2—1} = [n], where each undirected edge (i,7),0<i<j<n—1 
is present in the graph with probability c/n, independently for all n(n — 1)/2 
edges. Here c > 0 is a fixed constant, independent of n. A random r-regular 
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graph G,(n) is obtained by fixing a constant integer r > 2 and considering a 
graph selected uniformly at random from the space of all r-regular graphs on n 
nodes (graphs in which every node has degree r). A set of nodes V in a graph G is 
defined to be an independent set if no two nodes of V are connected by an edge. 
Let Z(n,c) and Z(n,r) denote the maximum cardinality of an independent set 
in G(n, c/n) and G,(n) respectively. Suppose the nodes of a graph are equipped 
with some non-negative weights W;,0 < i < n — 1 which are generated indepen- 
dently according to some common distribution F),(t) = P(W; < t),t > 0. Let 
Tu(n, c), Tu(n,r) denote the maximum weight of an independent set in G(n, c/n) 
and G,(n) respectively. 

Let M(n,c) and M(n,r) denote the maximum cardinality of a matching 
in G(n, c/n) and G,(n), respectively. It is known that G,(n),r > 3 has a full 
matching w.h.p., that is M(n,r) = n/2 (|n/2] for odd n) w.h.p. [JLROO]. If the 
edges of the graph are equipped with some non-negative random weights, then 
we consider instead the maximum weight of a matching M,,(n,c) and M,,(n,r) 
in graphs G(n, c/n), G,(n), respectively. The computation of Zw (n, c), Zy(n, 7), 
Mwy(n,c), My(n,7r) in the limit as n — oo is the main subject of the present 
paper. 

The asymptotic values of M(n,c) for all c were obtained by Karp and Sipser 
using a simple greedy type algorithm in [KS81]. The result extends to T(n, c) 
but only for c < e. It is an open problem to compute the corresponding limit for 
independent sets for the case c > e or even to show that such a limit exists [Ald]. 
Likewise it is an open problem to compute the corresponding limit in random 
regular graphs. 

The developments in this paper show that, surprisingly, proving the exis- 
tence and computation of the limits lim, Z(n,-)/n, limn M(-)/n is easier in the 
weighted case than in the unweighted case, at least for certain weight distribu- 
tions. In particular, we compute the limits for independent sets in G,(n),r = 
2,3,4 and G(n,e/n),c < 2e, when the node weights are exponentially dis- 
tributed, and we compute the limits for matchings in G,(n) and G(n, c/n) for 
all r,c, when the edge weights are exponentially distributed. It was shown by the 
first author [Gam04] that the limit lim, M(n,-)/n exists for every weight distri- 
bution with bounded support, though the non-constructive methods employed 
prevented the computation of the limits. 

Our method of proof is based on a powerful local weak convergence method 
developed by Aldous [Ald92], [AldO1], Aldous and Steele [ASO3], Steele [Ste02], 
further empowered by a certain local optimality observation derived in this pa- 
per. Local weak convergence is a recursion technique based on fixed points of 
distributional equations, which allows one to compute limits of some random 
combinatorial structures, see Aldous and Bandyopadhyay [AB] and Aldous and 
Steele [ASO3] for recent surveys on these topics. In particular, the method is used 
to compute maximum weight matching on a random tree, when the weights are 
exponentially distributed. The tree structure was essential in [ASO3] for cer- 
tain computations and the approach does not extend directly to graphs like 
G(n,c/n) with c > 1, where the convenience of a tree structure is lost due to 
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the presence of a giant component. It was conjectured in [ASO3] that a some 
long-range independence property might be helpful to deal with this difficulty. 
The present paper partially answers this qualitative conjecture in a positive 
way. We introduce a certain operator T acting on the space of distribution func- 
tions. We prove a certain local optimality property stating that, for example, 
for independent sets, whether a given node 2 belongs to the maximum weight 
independent set is asymptotically independent from the portion of the graph 
outside a constant size neighborhood of i, iff T? has a unique fixed point dis- 
tribution. Moreover, when T? does have the unique fixed point, the size of the 
extremal object (say maximum weight independent set) can be derived from a 
fixed point of an operator 7. The computations of fixed points is tedious, but 
simple in principle and the groundwork for that was already done in [AS03]. 
We hope that the long-range independence holds in other random combinatorial 
structures as well. In fact the issues of long-range independence were already 
considered by Aldous and Bandyopadhyay [AB], Bandyopadhyay [Ban03] using 
the notion of endogeny. This specific version of long-range independence turned 
out to be critical in Aldous [Ald01] for proving the ¢(2) limit of the random 
assignment problem. 

The issue of long-range independence (usually called correlation decay in- 
stead) of random combinatorial objects is addressed in a somewhat different, 
statistical physics context in Mossel [Mos03], Brightwell and Winkler [BW0O3], 
Rozikov and Suhov [RS03], Martin [Mar03], Martinelli, Sinclair and Weitz 
[MSW03], where independent sets (hard-core model) and other combinatorial 
models are considered on regular trees, weighted by the Gibbs measure. In a dif- 
ferent setting Talagrand [Tal03] proves a certain long-range independence prop- 
erty for the random assignment problem, where the usual min-weight matching 
is replaced by a partition function on the space of feasible matchings. He uses a 
rigorous mathematical version of the cavity method, which originated in physics, 
to prove that the spins are asymptotically independent as the size of the problem 
increases. The particular form of the long-range independence is similar to the 
one we obtain, and in fact the cavity method has some similarity with the local 
weak convergence method. which is also based on considering extremal objects 
(say independent sets) with one or several nodes excluded. Finally, we refer the 
reader to Hartmann and Weigt [HW01] who derive the same result as Karp 
and Sipser for independent sets using non-rigorous arguments from statistical 
physics. 

We finish this section with some notational conventions. Exp(j), Pois(A) de- 
note respectively exponential and Poisson distributions with parameters p, A > 
0,0 < z < 1. Most of the proofs are too lengthy to fit under the page limitation 
and we refer the reader to [GNS03] for proofs. 


2 Prior Work and Open Questions 


It is known and simple to prove that the largest independent sets are O(n) 
w.h.p. in both G(n,c/n) and G,(n) when c and r are any constants inde- 
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pendent from n. Moreover, it is known that in random cubic graphs, w.h.p., 
6 log(3/2) — 2 = .432... < liminf, Z(n, 3)/n < limsup,, Z(n, 3)/n < .4591. The 
lower bound is due to Frieze and Suen [FS94], and the upper bound is due 
to Bollobas [Bol81]. The upper bound is generalized for any r > 1 and uses 
a very ingenious construction of random regular graphs via matching and ran- 
dom grouping, [Bol80], [JLROO]. It is natural to expect that the following is true, 
which unfortunately remains only a conjecture, appearing in several places, most 
recently in [Ald] and [ASO3]. 


Conjecture 1 For every c > 0 andr > 3 the limits 


im EZ) jm EEN 
n= 00 TL MOO n 


exist. 


The existence of these limits also implies the convergence to the same limits 
w.h.p. by applying Azuma’s inequality, see [JLROO] for the statement and the 
applicability of this inequality. 

The limits lim, EZ(n, c)/n, lim, EM(n, c)/n are known to exist for for inde- 
pendent sets when c < e and for matching for all c and were first derived by 
Karp-Sipser [KS81]. Their results were strengthened later by by Aronson, Frieze 
and Pittel [APF98]. 

The algorithm underlying Karp and Sipser analysis unfortunately is not ap- 
plicable to random regular graphs. Moreover, if the edges or the nodes of the 
graph G(n,c/n) are equipped with weights then the Karp-Sipser algorithm can 
produce a strictly suboptimal solution and cannot be used in our setting of 
weighted nodes and edges. Also, when the edges of G,(n) are equipped with 
weights, the problem of computing maximum weight matching becomes non- 
trivial, as opposed to the unweighted case when the full matching exists w.h.p. 

In a somewhat different domain of extremal combinatorics the following result 
was established by Hopkins and Staton [HS82]. A girth of a graph is the size of 
the smallest cycle. It is shown in [HS82] that the size of a largest independent 
set in an n-node graph with largest degree 3 and large girth is asymptotically 
at least (7/18)n — o(n). The techniques we employ in this paper allow us to 
improved this lower bound. 


3 Main Results 


We begin by introducing the key technique for our analysis — recursive distri- 
butional equations and fixed point solutions. This technique was introduced by 
Aldous [Ald92], [AldO1] and was further developed in Aldous and Steele [ASO3], 
Steele [Ste02], Aldous and Bandyopadhyay [AB]. Let W be a non-negative ran- 
dom variable with a distribution function F,,(¢) = P(W < t). We consider four 
operators T = Tz,,Tz,c, Tm,r, TM, acting on the space of distribution functions 
F(t),t > 0,where c > 0 is a fixed constant and r > 2 is a fixed integer. 
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1. Given W distributed according to Fy, (we write simply W ~ Fw), and given 
a distribution function F = F(t), let B1, B2,..., Brp ~ F be generated inde- 
pendently. Then Tz, : F — F”, where F” is the distribution function of B’ 
defined by 


B’ = max(0, W — y Bj). (1) 
1<i<r 
2. Under the same setting as above, let By,...,Bm ~ F, where m is a random 


variable distributed according to a Poisson distribution with parameter c, 
independently from W, B;. Then TZ, : F — F’, where F” is the distribution 
function of B’ defined by 


B' = max(0,W — X` Bi), (2) 


i<i<m 


when m > 1 and B’ = W when m = 0. For simplicity we identify the sum 
above with zero when m = 0. 

3. Let Wi,...,Wr ~ Fw, By,..., Br ~ F. Then Tm r : F — F’, where F” is 
the distribution function of B’ defined by 


eq: RecursionMrB' = max (0, W; — B,). (3) 


from W;, Bi. Then Tm, : F — F’, where F” is the distribution function of 
B’ defined by 


f _ a z 
B = max (0,W: ~ Bi), (4) 
when m > 1 and B’ = Owhen m = 0. Again, for simplicity, we assume that 
the max expression above is zero when m = 0. 


A distribution function F is defined to be a fixed point distribution of an 
operator Tif T(F) = F. 

We now state the main result of this paper. Recall, that a distribution func- 
tion F(t) is defined to be continuous (atom free) if for every æ in its sup- 
port lim,o(F (x + €) — F(x — €)) = 0. Equivalently, for B ~ F and every 
x, P(B = x) =0. We use 1 {-} to denote the indicator function. 


Theorem 1. Let F be a continuous non-negative distribution function. For 
r > 1 if the operator T -1 has a unique fixed point distribution function F*, 
then, w.h.p. 


Ly(n, 
lim SAGE) = E[W 1{W — ) B; > 0}, (5) 
n n - 
1<i<r 
where W ~ Fy, Bi ~ F*, and W, B; are independent. When G,(n) is replaced by 


G(n,c/n), the same result holds for T = Tz «e, except the sum in the right-hand 
side of (5) is Decree B; and m ~ Pois(c). 
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Finally, the similar results hold for My(n,r) and M,(n,c) in G,(n) and 
G(n,c), forT = Tm,r-1 and T = Tmc, respectively, whenever the correspond- 
ing operator T is such that T? has the unique fixed point distribution F*. The 
corresponding limits are 


Maln, r) 1 
lim e] X. Wi1{W,- Bi = max (W; —B;)>0}, (6) 


1<i<r 
where W; ~ Fw, Bi ~ F*, and 
_ Mu(n,c) 1 
a ao we W: 1{W; — B: = max(W; - B;) >0}], (7) 


where W; ~ Fy, Bi ~ F*,m ~ Pois(c). 


The Theorem 1 is the core result of this paper. It will allow us to obtain several 
interesting corollaries, which we state below. The rest of the paper gives a coarse 
outline of the proof of Theorem 1. A detailed proof is found in [GNS03]. 


Theorem 2. Suppose the weights of the nodes and edges of the graphs G = 
Gr(n) and G = G(n,c) are distributed as Exp(1). Then 


L. T r- , has a unique fixed point distribution F* iff r < 4. In this case, w.h.p. 


Cp ce (8) 
n n 4 
where b is the unique solution of b = 1 — So In particular, w.h.p. 
lim a = = lim Luth 2) æ .6077, lim a ~ .4974, (9) 


2: TF has a unique fixed point distribution F* iff c < 2e. In this case, w.h.p. 


c(i — b) 


O Car 


me ) (10) 


lim 
m 


where b is the unique solution of 1—b = e780 -® In particular, when c = 2e, 
this limit is œ% .5517. 
3. Tirei has a unique fixed point F* for every r > 2. Moreover, w.h.p. 


lim 


n 


Hern = r(b"7} + o te“"(1—e7*(1—)))"""dt_— (11) 


= r| te™t(1 — e™*(1 —b))) dt, 
0 


where b is the unique solution of b = 1 — AD 
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4. Tii has a unique fixed point F* for all ce > 0. Moreover, for every c > 0 
w.h.p. 


oe) 
Hii Mau(n, c) = Llet- ae n | te~t—e(l—b)e* H (12) 
n n 2 0 
a c [ te t—2e(1—bye“*+e(1—b)7e 78 qt 
2 Jo 


where bis the unique solution of 1 — e~° = c(1 — b)?. 


The proof of Theorem 2, specifically the corresponding computations of the 
limits is quite lengthy and can be found in [GNSO3]. The expression in (11) 
involving integrals is similar to the one found in [AS03] for maximum weight 
matching on a tree. It is a pleasant surprise, though, that the answers for inde- 
pendent sets are derived in closed form. 

Part 2 of Theorem 2 leads to an interesting phase transition behavior. For 
Fo = Exp(1) our result says that the value of c = 2e is a phase transition 
point for the operator T2. where for c < 2e the operator has a unique fixed 
point distribution, but for c > 2e the fixed point distribution is not unique. It 
turns out (see Theorems 4, 5 below) that this phase transition is directly related 
to some long range independence/dependence property of the maximum weight 
independent sets in the underlying graph G(n, c/n). Interestingly, no such phase 
transition occurs for maximum weight matchings. 

The methods developed in this paper allow us the improve the following 
result of Hopkins and Staton [HS82]. Let G(n,r,d) denote the class of all (non- 
random) graphs on n nodes, with maximum degree r and girth at least d. For 
any G € G(n,r,d) let Z(G) denote the size of the largest independent set in G. 
Hopkins and Staton proved that lim infn,a mingegin,3,4) LZ(G)/n = 7/18 & .3887. 
Our techniques allow us to obtain the following improvement. 


Theorem 3. 


I(G 
liminf min (G) 
nd GEG(n3d) n 





> 3923. 


For the proof of this theorem see [GNS03]. 


4 Fixed Points of the Operator T? 
and the Long-Range Independence 


4.1 Maximum Weight Independent Sets and Matchings in Trees. 
Fixed Points of T? and the Bonus Function 


We start by analyzing operator T — any of the four operators introduced in the 
previous section. Given two distribution functions F3, F2 defined on [0, oo), we 
say that Fə stochastically dominates Fı and write Fy ~ Fy if F(t) > Fo(t) 
for every t > 0. A sequence of distribution functions Fn is defined to converge 
weakly to a distribution function F (written F, > F) if lim, F,(¢) = F(t) for 
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every t which is a point of continuity of F. All the convergence of distributions 
is understood here in a weak sense. 

Let 0 denote (for simplicity) the distribution function of a random variable X 
which is zero w.p.1. Let W™°* = maxi<i<r W; where W; ~ Fy, are independent. 
Let also W** = maxi<i<m Wi, where W; ~ Fuy are independent and m ~ 
Pois(c). Denote by Fw,- and Fw,c the distribution functions of W"®* and W™m*, 
respectively. 


Proposition 1. Fix r > 1,c > 0, a distribution function Fẹ„(t)},t > 0 and 
T = Tr, or Tr. As s > œ, the two sequences of distributions T?! (0), T”: (Fw) 
weakly converge to some distribution functions F,,,F'**, respectively, which are 
fixed points of the operator T?. For any distribution function Fy = Fo(t),t > 0, 
T?#(0) < T29+2(Fo) < T” (Ep) and THF) < T?8+3(Fo) < T28+1(0) for all 
E 

Ifthe operator T is such that T? has a unique fixed point (and F,, = F** = 
F*), then for any distribution function Fo = Fo(t), T (Fo), s = 1,2, ... converges 
to F* as s > œ. In particular, T*(0), T° (Fa) — F*. Moreover, F* is also the 
unique fixed point of T. When T = Tm, or T = Tm, the same result holds with 
Fw r and Fu c respectively replacing Fry. 


Proof. See [GNS03]. 


We now switch to analyzing the maximum weight independent set problem 
on a tree. The derivation here repeats the development in [ASO3] for maximum 
weight matching in random trees. We highlight important differences where ap- 
propriate. 

Suppose we have a (non-random) finite tree H with nodes 0,1,...,h = |H|— 
1, with a fixed root 0. The nodes of this tree are equipped with some (non- 
random) weights Wo, W1, ..., Wh => 0. For any node i € H, let H(i) denote the 
subtree rooted at 7 consisting of all the descendants of 7. In particular, H(O) = H. 
Let Zy) denote the maximum weight of an independent set in H(i) and let 
Buy) = Lay) - 5 j Tyg}, where the sum runs over nodes 7 which are children 
of i. If 7 has no children then Bg) is simply Zya) = Wi. Observe, that Byg) is 
also a difference between Zya and the maximum weight of an independent set 
in H(i), which is not allowed to use node 1. Clearly, 0 < Bua) < Wi. The value 
Bri) was considered in [ASO3] in the context of maximum weight matchings and 
was referred to as a bonus of a node i in tree H(i). W.l.g. denote by 1,...,m 
the children of the root node 0. 


Lemma 1. 
Broo) = max(0, Wo mi 5 Bgo) (13) 


1<i<m 


Moreover, if Wo > } i<cicm Puy (that is if Bo) > 0) then the maximum 
weight independent set must contain node 0. If Wo < Y i<icm Bai) then the 
maximum weight independent set does not contain the node 0. 


Proof. See [GNS03]. 
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A similar development is possible for maximum weight matching. Suppose 
the edges of the tree H are equipped with weights W; j. Let My,;;) denote the 
maximum weight of a matching in H(z), and let Byg) denote the difference 
between Myu) and the maximum weight of a matching in H(z) which is not 
allowed to include any edge in H(i) incident to i. Again 1,2,...,m are assumed 
to be the children of the root 0. 


Lemma 2. 
By = max(0, ymax (Wo, e Buiiy)). (14) 


Moreover, if Wo; — Bua) > Wow — Buy for all t +i and Wo; — Buy > 9, 
then every maximum weight matching contains edge (0,1). If Wo; — Bui < 0 
for alli =1,...,m, then every maximum weight matching does not contain any 
edge incident to 0. 


Proof. See [GNS03]. 


4.2 Long-Range Independence 


We now consider trees H of specific types. Given integers r > 2,d > 2 let H,(d) 
denote an r-regular finite tree with depth d. The root node O has degree r — 1, 
all the nodes at distance > 1,< d — 1 from the root have outdegree r — 1, 
and all the nodes at distance d from 0 are leaves. (Usually, in the definition of 
an r-regular tree, the root node is assumed to have degree r, not r — 1. The 
slight distinction here is done for convenience.) Also, given a constant ce > 0, 
a Poisson tree H(c,d) with parameter c and depth d is constructed as follows. 
The root node has a degree which is a random variable distributed according 
to Pois(c) distribution. All the children of 0 have outdegrees which are also 
random, distributed according to Pois(c). In particular, the children of 0 have 
total degrees 1+ Pois(c). Similarly, children of children of O also have outdegree 
Pois(c), etc. We continue this process until either the process stops at some depth 
d’ < d, where no nodes in level d’ have any children, or until we reach level d. 
In this case all the children of the nodes in level d are deleted and the nodes in 
level d become leaves. We obtain a tree with depth < d. We call this a depth-d 
Poisson tree. 

Let H = H,(d) or H(e,d). Suppose the nodes and the edges of H are 
equipped with weights W;,W;,;, which are generated at random independently 
using a distribution function Fẹ. Fix any infinite sequences Ù = (w1, w2,...) € 
(0,00) and b = (61, be,...) € {0,1}. For every i =1,2,...,d let 71, 72,..., iji 
denote the nodes of H in level i (if any exist for H(c,d)). When H = H,(d), ji = 
(r —1)*, of course. Let (Z|(b, w)) denote the maximum weight of an independent 
set V in H such that the nodes dj with b; = 1 are conditioned to be in V, 
nodes dj with b; = 0 are conditioned not to be in V, and the weights of nodes 
dj are conditioned to be equal to w; for 7 = 1,...,Ja. That is we are looking 
for maximum weight of an independent set among those which contain depth 
d leaves with b; = 1, do not contain depth d leaves with b; = 0, and with the 
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weights of the leaves deterministically set by w. For brevity we call it the maxi- 
mum weight of an independent set with boundary condition (b, #). For the case 
H = H (c,d), the boundary condition is simply absent when the tree does not 
contain any nodes in the last level d. (Zi(;;)|(b, @)) are defined similarly for the 
subtrees H(ij) spanned by nodes ij in level 7. Given again b, w, let (M{(b, w)) 
and (M #(i3) \(b, @)) denote, respectively, the maximum weight of a matching E 
in H and H;;, such that the edges incident to nodes dj are conditioned to be in 
E when b; = 1, edges incident to nodes dj are conditioned not to be in D when 
with 6; = 0, and the weights of the edges incident to nodes dj are conditioned 
to be equal to w;,j7 = 1,...,ja (of course, we refer to edges between nodes in 
levels d — 1 and das there is only one edge per each node in level d). 

For the case of independent sets, let (B|(b,)) denote the bonus of the root 
node 0 given the boundary condition (b, w). Namely, 


(B\(b,)) = (Z\(6,a))- SY) Zaanl(b,@)). 


£59 S91 


For the case of matchings, let (B |(b, w wW)) also denote the bonus of the root node 
0 given the boundary condition (b, ©). Namely, 


(BÈ, 5)) = (M|, 5))- ST (Maayl(6,0)). 


I1<j<ji 


It should be always clear from the context whether B is taken with respect to 
independent sets or matchings. 

The following theorem establishes the crucial long-range independence prop- 
erty for the maximum weight independent sets and matchings in trees H = 
H,(d), H(c,d) when the corresponding operator T? has a unique fixed point. It 
establishes that the distribution of the bonus (B/(6,w)) of the root is asymptot- 
ically independent of the boundary condition (b, Ū) as d becomes large. Recall 
our convention that O denotes the distribution of a random variable which is 
equal to zero with probability one. 


Theorem 4. Given a distribution function F, and a regular tree H = H,(d). 
Suppose the operator T? = TF 4 has the unique fixed distribution F*. Then 


sup |P((B|(b, ©)) < t) — F*(£)| — 0, (15) 
b.w 
as d — oo. Similar assertion holds for T = Tm r, T = Tz. and T = Tm,e. For 
the cases T = Tm r and Tm,c, Fw is replaced with Fy, and Fw,¢ respectively. 


Let us compare this result with the developments in ([ASO03]). In that paper 
maximum weight matching is considered on an n-node tree, drawn independently 
and uniformly from the space of all n™~? labelled trees. The notion of a bonus 
is introduced and the recursion (14) is derived. However, since a tree structure 
is assumed to begin with, there is no need to consider the boundary conditions 
(b, w). Here we avoid the difficulty of the non-tree structure by proving the long- 
range independence property via the uniqueness of fixed points of T?. 
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Proof. See [GNS03]. 


While it is not used in this paper, it is interesting that the uniqueness of 
the solution T?(F) = F is the tight condition for (15), as the following theorem 
indicates. 


Theorem 5. Under the setting of Theorem 5 suppose the operator T? has more 
than one fixed point distributions F*. Then for every such F* 


lim inf sup P((B\(b, )) < t) — F*(t)| > 0. (16) 


bw. 


Proof. See [GNS03]. 


4.3 Applications to Maximum Weight Independent Sets 
and Matchings in G,.(n) and G(n, c/n) 


The goal of the current section is to demonstrate that Theorem 4 allows us 
to reduce the computation of the maximum weight independent set and the 
maximum weight matching in random graphs to a much simpler problem of 
finding those in trees. We highlight this key message of the paper as the following 
local optimality property: if the operator T? corresponding to a maximum weight 
combinatorial object (independent set or matching) in a sparse random graph has 
a unique fixed point, then for a randomly selected node (edge) of the graph, the 
event “the node (edge) belongs to the optimal object” and the distribution of the 
node (edge) weight, conditioned that it does, asymptotically depends only on the 
constant size neighborhood of the node and is independent from the rest of the 
graph. In other words, when T? has a unique fixed point, the maximum weight 
independent sets and matchings exhibit a long-range independence property. 
Our hope is that similar local optimality can be established for other random 
combinatorial structures. A complete proof of Theorem 1 is given in [GNSO3]. 
Here we just give a high level idea. We observe that by symmetry E|Z,,(n,r)] = 
nE[Wo1{0 € V,}|,where V, is the independent set which achieves the maximum 
weight. We fix a large constant d and consider a size-d neighborhood of the 
node 1. With probability approaching unity this neighborhood is an r-regular 
tree. We then invoke Theorem 5 to argue that the quantity E[Wo1{0 € V,}| 
is determined almost exactly just by the d-neighborhood of the node 1 and is 
given asymptotically as E[Wo1 {Wo0 — $`; <;<, Bi > O} where B,,...,B, are iid 
distributed as F* and F* is the unique fixed point of the operator T. r-r 
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Abstract. The problem of estimating the k*” frequency moment F% for any non- 
negative k, over a data stream by looking at the items exactly once as they arrive, 
was considered in a seminal paper by Alon, Matias and Szegedy [1,2]. The space 
complexity of their algorithm is O(n!-«). For k > 2, their technique does not 
apply to data streams with arbitrary insertions and deletions. In this paper, we 
present an algorithm for estimating F, for k > 2, over general update streams 
whose space complexity is O(n) 7) and time complexity of processing each 
stream update is O(1). 

Recently, an algorithm for estimating Fk over general update streams with sim- 
ilar space complexity has been published by Coppersmith and Kumar [7]. Our 
technique is, (a) basically different from the technique used by [7], (b) is simpler 
and symmetric, and, (c) is more efficient in terms of the time required to process 
a stream update (O(1) compared with O(n a )). 


1 Introduction 


A data stream can be viewed as a sequence of updates, that is, insertions and deletions 
of items. Each update is of the form (l, +v), where, l is the identity of the item and v 
is the change in frequency of l such that |v| > 1. The items are assumed to draw their 
identities from the domain [N] = {0, 1,..., N— 1}. If v is positive, then the operation 
is an insertion operation, otherwise, the operation is a deletion operation. The frequency 
of an item with identity l, denoted by fj, is the sum of the changes in frequencies of 
l from the start of the stream. In this paper, we are interested in computing the kt? 
frequency moment Fk = >>, fF, for k > 2 and k integral, by looking at the items 
exactly once when they arrive. 

The problem of estimating frequency moments over data streams using randomized 
algorithms was first studied in a seminal paper by Alon, Matias and Szegedy [1,2]. 
They present an algorithm, based on sampling, for estimating Fk, for k > 2, to within 
any specified approximation factor € and with confidence that is a constant greater than 
1/2. The space complexity of this algorithm is s = Õ(n!- È) (suppressing the term 
4) and time complexity per update is O(n!-®), where, n is the number of distinct 
elements in the stream. This algorithm assumes that frequency updates are restricted to 
the form (J, +1). 

One problem with the sampling algorithm of [1,2] is that it is not applicable to 
streams with arbitrary deletion operations. For some applications, the ability to handle 
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deletions in a stream may be important. For example, a network monitoring applica- 
tion might be continuously maintaining aggregates over the number of currently open 
connections per source or destination. 

In this paper, we present an algorithm for estimating Fk, for k > 2, to within an 
accuracy of (1 + €) with confidence at least 2/3. (The method can be boosted using the 
median of averages technique to return high confidence estimates in the standard way 
[1,2].) The algorithm handles arbitrary insertions and legal deletions (1.e., net frequency 
of every item 1s non-negative) from the stream and generalizes the random linear com- 
binations technique of [1,2] designed specifically for estimating F2. The space com- 
plexity of our method is O(n'~ F=1) and the time complexity to process each update is 
O(1), where, functions of k and e that do notinvolve n are treated as constants. 

In [7], Coppersmith and Kumar present an algorithm for estimating Fk over general 
update streams. Their algorithm has similar space complexity (1.e., O(n) rT )) as the 
one we design in this paper. The principal differences between our work and the work 
in [7] are as follows. 


1. Different Technique. Our method constructs random linear combinations of the 
frequency vector using randomly chosen roots of unity, that is, we construct the 
sketch Z = fıxı, where, x; is a randomly chosen kt” root of unity. Coppersmith 


and Kumar construct random linear combinations C = f;a;, where, for l € [N], 


Tı = —1/n} or 1 —1/n!~&7 with probability 1 — 1/n? TE and 1/n! 7 


respectively. 

2. Symmetric and Simpler Algorithm. Our technique is a symmetric method for all 
k > 2, and is a direct generalization of the sketch technique of Alon, Matias and 
Szegedy [1,2]. In particular, for every k > 2, E [Re Z f] = Fk. The method of 
Coppersmith and Kumar gives complicated expressions for estimating Fp, for k > 
4. For k = 4, their estimator is C4 — Ban F2 (where, Ban œ n~4/3(1 — n™?/3)2), 
and requires, in addition, an estimation of F> to within an accuracy factor of (1 + 
n—1/3). The estimator expression for higher values of k (particularly, for powers 
of 2) are not shown in [7]. These expressions require auxiliary moment estimation 
and are quite complicated. 

3. Time Efficient. Our method is significantly more efficient in terms of the time taken 
to process an arrival over the stream. The time complexity to process a stream 
update in our method is O(1), whereas, the time complexity of the Coppersmith 


Kumar technique is O(n!~ 1*1 J. 


The recent and unpublished work in [11] presents an algorithm for estimating Fk, for 
k > 2 and for the append only streaming model (used by [1, 2]), with space complexity 
O(n!~ T), Although, the algorithm in [11] improves on the asymptotic space com- 
plexity of the algorithm presented in this paper, it cannot handle deletion operations 
over the stream. Further, the method used by [11] is significantly different from the 
techniques used in this paper, or from the techniques used by Coppersmith and Kumar 


[7]. 


Lower Bounds. The work in [1,2] shows space lower bounds for this problem to be 
Q(n}—5/*), for any k > 5. Subsequently, the space lower bounds have been strength- 
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ened to 2(e?2n!-(@+6)/F) for k > 2, € > 0, by Bar-Yossef, Jayram, Kumar and Sivaku- 
mar [3], and further to 2(n!~?/*) by Chakrabarti, Khot and Sun [5]. Saks and Sun [14] 
show that estimating the Lp distance d between two streaming vectors to within a factor 
of dê requires space 2(n!—?/P-4°), 


Other Related Work. For the special case ofcomputing F>, [1,2] presents an O(log n+ 
log m) space and time complexity algorithm, where, m is the sum of the frequencies. 
Random linear combinations based on random variables drawn from stable distributions 
were considered by [13] to estimate Fp, for O < p < 2. The work presented in [9] 
presents a sketch technique to estimate the difference between two streams based on 
the Lı metric norm. There has been substantial work on the problem of estimating Fo 
and related metrics (set expression cardinalities over streams) for the various models of 
data streams [10,1,4,12]. 

The rest of the paper is organized as follows. Section 2 describes the method and 
Section 3 presents formal lemmas and their proofs. Finally we conclude in Section 4. 


2 An Overview of the Method 


In this section, we present a simple description of the algorithm and some of its proper- 
ties. The lemmas and theorems stated in this section are proved formally in Section 3. 
Throughout the paper, we treat k as a fixed given value larger than 1. 


2.1 Sketches Using Random Linear Combinations of k*” Roots of Unity 


Let x be a randomly chosen root of the equation zë = 1, such that each of the k roots 
is chosen with equal probability of 1/k. Given a complex number z, its conjugate is 
denoted by Z. For any j, 1 < j < k, the following basic property holds, as shown 
below. 


E[r] = E[7] = 


0 aufil<j7<k 
a (1) 


1 ifj=k. 


Proof. Let j = k. Then, E[x?] = E|z*] = E[1] = 1, since, x is a root of unity. 

Let1 < j < k and letu be the elementary kt? root of unity, that is, u = e?” V71F, 
=F De =Ed ul)! = a 
= = k (1—w) 
where, the last equality follows from the sum of a geometric progression in the complex 
field. Since u* = 1, it follows that u3* = 1. Further, since u is the elementary kt” root 
ofunity, uf = e?"ivV-1/k 4 1 for 1 < j < k. Thus, the expression (1 — u?*)/(1 — 
u) = 0. Therefore, E[z?] = 0, forl <j < k. 

The conjugation Op alor is a 1-1 and onto operator in the field of complex numbers. 
Further, if x is a root of zë = 1, then, Z€ = x¥ = I = 1, and therefore, Z is also a kt” 


root of unity. Thus, the conjugation operator, applied to the group of kth roots of unity, 
results in a permutation of the elements in the group (actually, it is an isomorphism). It 


ee 
ee 
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therefore follows that the sum of the 7*” powers of the roots of unity is equal to the sum 
of the j*” powers of the conjugates of the roots of unity. Thus, E[z?] = E[z/]. J 


Let Z be the random variable defined as Z = diel N] fiz,. The variable x;, for 
each! € [N], is one of a randomly chosen root of zë = 1. The family of variables {2;} 
is assumed to be 2k-wise independent. The following lemma shows that Re Z* is an 
unbiased estimator of Fẹ. Following [1,2], we call Z as a sketch. The random variable Z 
can be efficiently maintained with respect to stream updates as follows. First, we choose 
a random hash function 8 : [N] — [k] drawn from a family of hash functions that is 2k- 
wise independent. Further, we pre-compute the kt” roots of unity into an array A[1..k] 
of size k (of complex numbers), that is, Afr] = e?™"V~1/*, for r = 1,2,...,k. For 
every stream update (l, v), we update the sketch as follows. 


Z=Z+0-Al[6(l)| 


The space required to maintain the hashfunction @ = O(k), and the time required for 
processing a stream update is also O(k). 


Lemma 1. E[Re Z*] = Fy. 
As the following lemma shows, the variance of this estimator is quite high. 
Lemma 2. Var[Re Z*| = O(k?* FY). 


This implies that Var [Re Z*]/(E[Re Z*|)? = O(FY/F?), which could be as large 
as n*—2_To reduce the variance we organize the sketches in a hash table. 


2.2 Organizing Sketches in a Hash Table 


Let d: {0,1,..., N—1} — [B] be a hash function that maps the domain {0,1,..., N— 
1} into a hash table consisting of B buckets. The hash function œ is drawn from a 
family of hashfunctions # that is 2k-wise independent. The random bits used by the 
hash family is independent of the random bits used by the family {ri }1€{0,1,..., N—1} OF, 
equivalently, the random bits used to generate @ and @ are independent. The indicator 
variable yz,4, for any domain element! € {0,1,...,N — 1}] and bucket b € [B], is 
defined as y; = 1 if d(/) = b and y;,» = Q otherwise. Associated with each bucket b is 
a sketch Z, of the elements that have hashed to that bucket. The random variables, Y, 
and Z, are defined as follows. 


Z= Do fesem Ye=ReZf, and Y= 9Y 
l bE[B] 


Maintaining the hash table of sketches in the presence of stream updates is analogous 
to maintaining Z. As discussed previously, let 8 : {0,1,..., N — 1} — [k] denote a 
random hash function that is chosen from a 2k-wise independent family of hash func- 
tions (and independently of the bits used by ¢), and let Af1 . . . k] be an array whose jt” 
entry is e? ™7-V—1/F for j = 1,...,k. For every stream update (l, v), we perform the 
following operation. 

Zo) = Zem +v: AP] 
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The time complexity of the update operation is O(k). The sketches in the buckets except 
the bucket numbered (l) are left unchanged. 

The main observation of the paper is that the hash partitioning of the sketch Y into 
{Yo bejn] reduces the variance of Y significantly, while maintaining that E[Y] = Fy. 
This is stated in the lemma below. 


Lemma 3. Let B < 2n!~%. Then, Var[Y] = O(F2n*-?/B*-}). 


A hash table organization of the sketches is normally used to reduce the time com- 
plexity of processing each stream update [6,8]. However, for k > 2, the hash table 
organization of the sketches has the additional effect of reducing the variance. 

Finally, we keep sı independent copies Y [0], ..., Y [sı — 1} of the variable Y. The 
average of these variables is denoted by Y; thus Var[Y] = (1/s1)Var[Y¥]. The re- 
sult of the paper is summarized below, which states that Y estimates Fy to within an 
accuracy factor of (1 + €) with constant probability greater than 1/2 (at least 2/3). 


Theorem 1. Letn! TTT < B < 2-n!7 7 ands; = 6-2" - k8*/e?. Then, 
Pr {|Y — Fy! > eFp} < 1/3. 


The space usage of the algorithm is therefore Õ(B - s1) = O(n!) bits, since 
a logarithmic overhead is required to store each sketch Z». To boost the confidence of 
the answer to at least 1 — 272(°2), a standard technique of returning the median value 
among Sg such average estimates can be used, as shown in [1,2]. 


The algorithm assumes that the number of buckets in the hash table is B, where, 
1 


nT < B <Q. ni- et, Since, in general, the number of distinct items in the 
stream is not known in advance, one possible method that can be used is as follows. 
First estimate n to within a factor of (1 + £) using an algorithm for estimating Fo, 
such as [10,1,2,4]. This can be done with high probability, in space O(logN). Keep 
2 log N+4 group of (independent) hash tables, such that the i” group uses B; = [2*/2] 
buckets. Each group of the hash tables uses the data structure described earlier. At the 
time of inference, first n is estimated as 7%, and, then, we choose a hash table group 
indexed by į such that ¿ = 2- [(1 — zł) log (8 - ñ/T)|. This ensures that the hash 


table size B; satisfies ni- r= < Bi <2- nim FET, with high probability. Since, the 
number of hash table groups is 2 - log N, this construction adds an overhead in terms 
of both space complexity and update time complexity by a factor of 2 - log N. In the 
remainder of the paper, we assume that n is known exactly, with the understanding that 
this assumption can be alleviated as described. 


3 Analysis 


The j** frequency moment of the set of elements that map to bucket b under the hash 
function ®, is a random variable denoted by F} b. Thus, F}, = >>, f? Yı». Further, since 
every element in the stream hashes to exactly one bucket, 5°, F}, = Fj. We define hyp, 
for J € {0,1,..., N — 1} and b € [B] to be Ay, = fi - yi». Thus, Fy, = do, ce for 
j z1. 
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Notation: Marginal Expectations. The random variables, Y, {Y,}ep are functions of 
two families of random variables, namely, x = {x;} 1€{0,1,...,V—1}» used to generate the 
random roots of unity, and y = {yz}, l € {0,1,..., N —1} and b € [B], used to map 
elements to buckets in the hash table. Our independence assumptions imply that these 
two families are mutually independent (1.e., their seeds use independent random bits), 
that is, Pr {x = u and y = v} = Pr {x =u} - Pr {y = v} Let W = WG, y) bea 
random variable that is a function of the random variables in x and y. For a fixed random 
choice of y = yo, Ex [Ww] denotes the marginal expectation of W as a function of y. 
That is, Ex [W] = >, W (u, yo)Pr {x = u}. It follows that E[W] = Ey [Ex [W]]. 


Overview of the Analysis. The main steps in the proof of Theorem 1 are as follows. 
In Section 3.1, we show that Ex [Y] = Fx. In Section 3.2, we show that E [Re Z "i < 
k2* FF. In Section 3.3, using the above result, we show that Ex bad Lig Fy = 


Section 3.4 shows that Ey [F},] < (2/B + 2* - n*~?/B*) F? and also concludes the 
proof of Theorem 1. Finally, we conclude in Section 4. 


Notation: Multinomial Expansion. Let X be defined as X = >), {0,1,...,N—1} 2b 
where, a; > 0,for l € {0,1,..., N —1}.Then, X* can be written as 


k 


k M 
Xf = ) ) ` ar ap a? 
€1€2°°'°es 1 2 3 


s= e1 +t es =k, e1 >00,- eg >0 Liclg<<l, 


where, sis the number of distinct terms in the product and e; 1s the exponent of the 
th product term. The indices l; are therefore necessarily distinct, l; € {0,1,...,N— 
1},¢ = 1,2,...,s. For easy reference, the above equation is written and used in the 


following form. 
Xt= X ce) >> (e). (2) 


s,e:Q(e,s) 1:R{e,l,s) j=l 

where, Q(e,s) =1<s < kand e = (€}, €2,...,€s)iS s-dimensional and dj=1 i= = 
k; R(e,l,s) = 1 = (4, l2,...,1,) is s- dimensional andO <ha <lh <e < |, 
N —1; and the multinomial coefficient Cie) = (a ee In this notation, the slow 
inequality holds . 

> [e < 11 dat" (3) 

1:R(e,l,s) j=1 

By setting n = k, and ay = ag =--- = ay = 1, we obtain, 


kk = 3 C(e) (‘) > >, C(e) 


By squaring the above equation on both sides, we obtain that k7* = (Ze, C(e)(£))? 
ee C?(e). We therefore have the following inequalities. 


So Cle) <k*, XO (e) <k”. (4) 
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3.1 Expectation 


In this section, we show that E[Re Z] = Fk, thereby proving Lemma 1, and that 
Bey =F 


Proof (of Lemma 1). Since the =“ of variables x;’s is k-wise independent, therefore 
j=l 


Applying equation (2) to Z* = i fix;)* and using linearity of expectation and k- 
wise independence property of x;’s, we obtain 


BIZJ= E ee S Te) 15 J) 


s,e:Q(e,s) l:R(e,l,s) j=l 
Using equation (1), we note that the term ([];_, E ae $ |) = 0, if s > 1, since in this 


case, e; < k, foreach 7 = 1,...,s. Thus, the above summation reduces to 
-Dir 
l 
Since Fy is real, E [Re Z*] is also Fẹ, proving Lemma 1. o 


Lemma 4. Suppose that the family of random variables {x1} is k-wise independent 
Then, Èx [Y] = Fk and Ex |Y] = E[Y] — Fp. 


Proof. We first show that E, [Yo] = Fry. Ex [ZE] = BaO fiy oxi )"] = 
Ex hı bz1)"], by letting hib = fi - yi. By an argument analogous to the proof of 
Lemma 1, we obtain Ex [(5>, hox) "] = Ei hf, = Si fot = Vi five = Feo, 
(since yj,p’s are binary variables). Since Fẹ, is always real, Ex [Yp] = Ex [Re Z] = 
Fi, p. Finally, Ex [Y] = E, [S Y»] =>) E [Y] = 5°, Fk, = Fp, since each ele- 
ment is hashed to exactly one bucket. Further, E[Y] = Ey [Ex [Y ]] = Ey [Fk] = Fe. 

LJ 


3.2 Variance of Re Z* 
In this section, we estimate the variance of Re Z* and derive some simple corollaries. 
Lemma 5. Let W = Re (3>, aix)". Then, Var[W] < k?*(3>, a?)*. 


Proof. Let X = (37, axı)". Then, Var[W] = E[W?] — (B[W])? < E[XxX] — 
(E [W] )?. Using equation (2), for X, X, we obtain the following. 


x= Yee) D e ) Cs 


s,e:Q(e,3) 1:R(e,l,s) j=l 


S> ce) > JI ož )-(T] 2%.) 


t,g:Q(g,t) 1:R(g,m,t) j/=1 j'=1 
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Multiplying the above two equations, we obtain 


XX= SO SO ee) ce ` 


s,e:Q(e,s) t,g:Q(g,t) 1: R(e,1,s) i: R(g,m,t) 
8 t s 
ei). Iji N, = 95 
Tf) (TT 2%) - EL 2) (I 2%). 
j=l j=l j=l j=l 


The general form of the product of random variables that arises in the multinomial 
expansion of X X is ([]5.1 x) Ta Tm ). Since the random variables x;’s are 2k- 
wise independent, using equation (1), it follows that, 


1 fs=t= igg =k 


eje 11,1 = = ifs=t,t>1l,e=gandl!=m 
0 otherwise. 


This directly yields the following. 


- 5 o y Tot 


e,3:Q(e,s) 1:R(e,k,s) 7=1 


< Xe) TH Ga by equation (3) 
j=1 l 
< J 07(e) IOJ aN since $ a,” < (X a)” 
e,s i A i r 
7 SNG. 07(e)), since ja =k 
l e,s Jai 


< > a?)" .k?* by equation (4). O 
l 


By letting a; = fı, lL € {0,1,2..., N — 1}, Lemma 5 yields 
Var [Re Z*] = Var [Re ($ fizi)*] < k** FY, (5) 
l 


which is the statement of Lemma 2. By letting a; = hib = fi - Yı», where, b is a fixed 
bucket index, and! € {0,1,2..., N — 1}, yields the following equation. 


Ex [Y}] < k” Ff, ford € [B]. (6) 


3.3 Var [Y]: Vanishing of Cross-Bucket Terms 


We now consider the problem of obtaining an upper bound on Var[Y]. Note that 
Var|Y] = Ey[Ex[Y?]] — (Ey[Ex[Y]])?. ¿From Lemma 4, Ey [Ex [Y]] = Fr. 
hus, 
Var[Y] = Ey (E,[Y*]] — FP . (7) 
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Lemma 6. Var [Y] —k" E; [Fo »|, assuming independence assumption I. 


Proof. Ex [Y?] = E [$ Y%)?] = Ex[o, ¥? + Daath YoYo] = Pp Ex LY] + 
er Ex [Ya Ya] s 

We now consider Ex [Ya Y] , for a # b. Recall that Ya = Re Z* (and analogously, 
Y, is defined). For any two complex numbers z, w, (Re z)(Re w) = (1/2)Re (z(w + 
w)). Thus, Ya Yp = (Re Z*)(Re Zy) = (1/2)Re (ZE ZF + ZE ZĘ). 

Let us first cono Ex [Za Z sl The general term involving product of random 
variables Is (L= fG) (TI; ==} fia i) (T];_ ==] Ylj,a x): JE AME Ym; b aa Je Con- 
sider the last two terms in the above expression, that 1s, (I=: Yla T )o 
(isi Ym;,b * Ta) For any 1 < j < sand 1 < j’ < t, it is not possible that 
l; = mj, that is, the same element whose index is given by 1; = m; cannot si- 
multaneously hash to two distinct buckets, a and b (recall that a Æ b). By 2k-wise 
independence, we therefore obtain that the only way the above product term can be non 
zero (i.e., 1) on expectation, is that s = t = 1 and therefore, e = k and gı = k. Thus, 
we have E[Z3Z5] = Dim hahmo = Fk,a Frb. 

Using the same observation, it can be argued that Ex [ZE ZE] = Fk,aFk,b. It fol- 
lows that Ex [(1/2)(ZSZf + Z'Z§))| = Fk,aFk,b, which is a real number. Therefore 
Ex [Re (1/2)(Z* ZF + ZE ZE)] = Fk a Fk o = Ex (Ya Yo]. 

By equation (7), Var[Y] = Ey [Ex[Y?]] — F?. Further, from Lemma 4, Fk = 
Ey [>>, Fk,»]- We therefore have, 


Var |Y] = Ey [Ex [Y°] - os F.,b) 1 


= Ey DE [Ye] IER] [Ya Ys] — (È Fe) | 


ab 
= = By | [2 Ex LY | + S Fr, afk, b — (È Fra) b f by above argument 
axb 
= E, Snl Ys] = 9 Eke] < Ey Sn Yol] 
b b b 
< E, D k?* Ff, ], by equation (6) o 
b 
3.4 Calculation of E |F} » | 
Given a t-dimensional vector e = (e€;,...,é¢) such that e; > 0, for 1 <i < t and 


Yj e; = k, we define the function w(e) as follows. Without loss of generality, let 
the indices e; be arranged in non-decreasing order. Let r = r(e) denote the largest 
index such that ep < k/2. Then, we define the function ¢(e) as follows. 


we) — now jai i 2€3/k) /B' 


The motivation of this definition stems from its use in the following lemma. 
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Lemma 7. Suppose er ej = k and e; > 0,for j =1,...,t. Then, 1s Poe, S 
p(e)- FÈ- Bt. 


Proof. From [1, 2], F; < ni-i/k RIE if j < k and F} < FIF ifj > k. Thus, 


t r t r t 
| | Fae; = | Poe, | | Foe; = | | n e;/ Pr if l F; i/ 
j=l j=1 


j=r+1 j=1 j=r4+ 1 


r . a 28; k s 
= nb j= (1-2e;/W phia E _ yey. Bt. FR, since Y` e; = kD 
j 


The function % satisfies the following property that we use later. 


Lemma 8. If B < 2+ n)~, then, W(e) < max (2/B,2* -n*-2/BF), 


Proof. Let e be a t-dimensional vector. If t = 1, p(e) = 1/B. If t = r, then (e) = 
ni-2 (Bt < 2F . nF-2/B* Ift >r + 2, then y(e) = (2¢/B*) nt (E-r) +d 2e;/k) < 
2t ont? / Bt < 2* .n*-2/B*. Finally, lett = r + 1. Then, 


p(e) 2790 yt-1-2h eik] Bt < gt, pe ee BF. 
since >;= €j 27 = t — 1. Thus, 


wie) < v(e)(2- n! k (Bye < iz ne ae DE BS ni~& /B)k-t 2 
Oh . nk-2-(=2)/k / pk < 9% . nk=2/ BR 


where, the first inequality follows from the assumption that B < n!~% and the second 
inequality follows because t > 2. O 


Lemma 9. Let B <2-n!~*. Then, E|FY,| < k* F2(2/B + 2* .nk-2/B*), 


Proof. For a fixed b, the variables 3,4 are k-wise independent. Fz,» is a linear function 
of yi,5- Thus, sg p is a Symmetric multinomial of degree k, as follows. 


Foe = (Di fim)’ => Cle) J, FO Fie Yab Yab Yab - 
l 8,8 


Iy<ig<---l, 
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Taking expectations, and using k-wise independence of the y;,p’s, we have, 


E [Fa] meen) ` JEE ++ fE E [Yn b Yiz, * Yia] 


li <lo<:--<l, 
-Eceo E f Blua] Blus] Blu 
li<la <e <ls 
| 1 
=$ Ce) J Hely’ g since, Ely,6] = > 
S, li<la< <le B B 
< 2l) -(1/B*) - [e < 206) JeF, by Lemma 7 


< sce . Fe. amad . ak-2/ BR) by Lemma 8 
<k*. F?2.(2/B+2* .n*-?/B*), a j< k" o 


Combining the result of Lemma 6 with Lemma 9, we obtain the following bound on 
Var[Y]. 
Var [Y ] < k3* Peo (24 J AAB: (8) 


Recall that Y is the average of sı independent estimators, each calculating Y. The main 
theorem of the paper now follows simply. 


Proof (of Theorem 1). By Chebychev’s inequality, Pr {|Y — Fk| > Fk} < 
Var [Y | /(e? F2). Substituting Equation (8), we have Var|Y]/(e?- F) < 1/3. o 


4 Conclusions 


The paper presents a method for estimating the k‘? frequency moment, for k > 2, 
of data streams with general update operations. The algorithm has space complexity 
Ö(n! r% )) and is based on constructing random linear combinations using randomly 
chosen k*t? roots of unity. A gap remains between the lower bound for this problem, 
namely, O(n!—?/*), for k > 2, as proved in [3,5] and the complexity of a known 
algorithm for this problem. 


References 


1. Noga Alon, Yossi Matias, and Mario Szegedy. “The Space Complexity of Approximating the 
Frequency Moments”. In Proceedings of the 28th Annual ACM Symposium on the Theory of 
Computing STOC, 1996, pages 20-29, Philadelphia, Pennsylvania, May 1996. 

2. Noga Alon, Yossi Matias, and Mario Szegedy. “The space complexity of approximating 
frequency moments”. Journal of Computer Systems and Sciences, 58(1):137-147, 1998. 

3. Ziv Bar-Yossef, T.S. Jayram, Ravi Kumar, and D. Sivakumar. “An information statistics ap- 
proach to data stream and communication complexity”. In Proceedings of the 34th ACM 
Symposium on Theory of Computing (STOC), 2002, pages 209-218, Princeton, NJ, 2002. 


380 


10. 


11. 


12. 


Sumit Ganguly 


. Ziv Bar-Yossef, T.S. Jayram, Ravi Kumar, D. Sivakumar, and Luca Trevisan. “Counting 


distinct elements in a data stream’. In Proceedings of the 6th International Workshop on 
Randomization and Approximation Techniques in Computer Science, RANDOM 2002, Cam- 
bridge, MA, 2002. 


. Amit Chakrabarti, Subhash Khot, and Xiaodong Sun. “Near-Optimal Lower Bounds on the 


Multi-Party Communication Complexity of Set Disjointness”’. In Proceedings of the 18th An- 
nual IEEE Conference on Computational Complexity, CCC 2003, Aarhus, Denmark, 2003. 


. Moses Charikar, Kevin Chen, and Martin Farach-Colton. “Finding frequent items in data 


streams”. In Proceedings of the 29th International Colloquium on Automata Languages and 
Programming, 2002. 


. Don Coppersmith and Ravi Kumar. “An improved data stream algorithm for estimating fre- 


quency moments”. In Proceedings of the Fifteenth ACM SIAM Symposium on Discrete Al- 
gorithms, New Orleans, LA, 2004. 


. G. Cormode and S. Muthukrishnan. “What’s Hot and What’s Not: Tracking Most Frequent 


Items Dynamically”. In Proceedings ofthe Twentysecond ACM S$IGACT-SIGMOD-SIGART 
Symposium on Principles of Database Systems, San Diego, California, May 2003. 


. Joan Feigenbaum, Sampath Kannan, Martin Strauss, and Mahesh Viswanathan. “An Ap- 


proximate L+-Difference Algorithm for Massive Data Streams”. In Proceedings of the 40th 
Annual IEEE Symposium on Foundations of Computer Science, New York, NY, October 
1999. 

Philippe Flajolet and G.N. Martin. “Probabilistic Counting Algorithms for Database Appli- 
cations”. Journal of Computer Systems and Sciences, 31(2):182-209, 1985. 

Sumit Ganguly. “A bifocal technique for estimating frequency moments over data streams”. 
Manuscript, April 2004. 

Sumit Ganguly, Minos Garofalakis, and Rajeev Rastogi. “Processing Set Expressions over 
Continuous Update Streams”. In Proceedings of the 2003 ACM SIGMOD International Con- 


ference on Management of Data, San Diego, CA, 2003. 


13. 


14. 


Piotr Indyk. “Stable Distributions, Pseudo Random Generators, Embeddings and Data 
Stream Computation”. In Proceedings of the 41st Annual IEEE Symposium on Foundations 
of Computer Science, pages 189-197, Redondo Beach, CA, November 2000. 

M. Saks and X. Sun. “Space lower bounds for distance approximation in the data stream 
model”. In Proceedings of the 34th ACM Symposium on Theory of Computing (STOC), 2002, 
2002. 


Fooling Parity Tests with Parity Gates 


Dan Gutfreund™* and Emanuele Viola”** 


' School of Computer Science and Engineering 
The Hebrew University of Jerusalem, Israel, 91904 
danig@cs.huji.ac.il 
* Division of Engineering and Applied Sciences 
Harvard University, Cambridge, MA 02138 


viola@eecs.harvard.edu 


Abstract. We study the complexity of computing k-wise independent 
and e-biased generators G : {0,1}" — {0,1}”. Specifically, we refer 
to the complexity of computing G explicitly, i.e. given x € {0,1}” and 
i € {0,1}'°&™ computing the i-th output bit of G(x). [MNT90] show that 
constant depth circuits of size poly(n) cannot explicitly compute k-wise 
independent and e-biased generators with seed length n < gles) m 

In this work we show that DLOGTIME-uniform constant depth circuits 
of size poly(n) with parity gates can explicitly compute k-wise inde- 
pendent and e-biased generators with seed length n roughly logm < 
gles" m Tn some cases the seed length of our generators is optimal up 
to constant factors, and in general up to polynomial factors. To obtain 
our results, we show a new construction of combinatorial designs, and we 
also show how to compute, in DLOGTIME-uniform ACo, random walks 
of length log* n over certain expander graphs of size 2”. 


1 Introduction 


The notion of pseudorandom generators (PRGs) is central to the fields of Com- 
putational Complexity and Cryptography. Informally, a PRG is an efficient de- 
terministic procedure that maps a short seed to a long output, such that certain 
tests are fooled by the PRG, 1.e. they cannot distinguish between the output of 
the PRG (over a random input) and the uniform distribution. More formally, a 
PRG G: {0,1}” — {0,1}™ fools a test M : {0,1} — {0,1} if 


Pr {[M(y)=1]- Pr |[M(G =1||<e. 

co De oe 
One popular class of tests M : {0,1}™ — {0,1} are parity ones (sometimes 
called linear tests). These are the tests that, given z € {0,1}, only XOR fixed 
subsets of the bits of z. A generator is called e-biased if it fools such tests. 


* Research supported in part by the Leibniz Center, the Israel Foundation of Science, 
a US-Israel Binational research grant, and an EU Information Technologies grant 
(IST-FP5). 

= Research supported by NSF grant CCR-0133096 and US-Israel BSF grant 2002246. 


K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 381-392, 2004. 
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Another popular type of tests are those whose value, given z € {0,1}™, depends 
only on & fixed bits of z, but on those bits is arbitrary. A generator is called 
(e, k)-wise independent if it fools such tests. An important special case of (e, k)- 
wise independent generators are those where e = 0, i.e. every k fixed output 
bits of the generator are uniform over {0,1}*. Such generators are called k- 
wise independent. ¢-biased and k-wise independent generators have found several 
applications in Complexity Theory and Cryptography. For a discussion of these 
generators we refer the reader to the excellent book by Goldreich [Gol99]. 


In this paper we study the following general question: what are the mini- 
mal computational resources needed to compute k-wise and €-biased generators? 
Throughout this work, when we refer to the complexity of a generator G, we 
actually refer to the complexity of computing the i-th bit of G(x) given a seed 
x € {0,1}” and an index i € {0,1}'°8™. This is a more refined notion of complex- 
ity, which is especially adequate for generators having logarithmic seed length 
(i.e. n = logm), as is the case with k-wise independent and e-biased generators 
(see below). Generators for which we can efficiently compute the 7-th output bit 
of G(x), given x € {0,1}" andi € {0,1}'°™, are called explicit. Explicitness 
plays a crucial role in many applications where only some portion of G(x) is 
needed at each time, and the application runs in time polynomial in the seed 
length of the generator (rather than in its output length). Such applications 
range from constructing hash functions to Probabilistically Checkable Proofs. 

k-wise independent and e-biased generators, G : {0,1}" — {0,1}™, explicitly 
computable in time poly(n) are known with seed length logarithmic in the output 
length, ie. n = O(logm) [CG89,ABI86,NN90,AGHP92] (we ignore here other 
parameters, we will be more accurate later). On the other hand, Mansour, Nisan 
and Tiwari [MNT90] showed! that any k-wise independent or e-biased generator 
G : {0,1}" — {0,1} explicitly computable by constant depth circuits (AC) of 

: log?) m 
size poly(n) must have seed length n at least 2°°8 > logm. 

In this paper we ask the following technical question: Can constant depth 
circuits with parity gates (ACo|®]) of size poly(n) explicitly compute k-wise 
independent and ¢-biased generators, G : {0,1}" — {0,1}™, with seed length 
n ~ logm? (Where parity is the function Q(z) := $; z; mod 2). 

We give an affirmative answer by showing several constructions of such gener- 
ators explicitly computed by ACo[®] circuits of size poly(n) (our results are dis- 
cussed below). All our circuits are DLOGTIME-uniform. Informally this means 
that each gate (resp. edge) in the circuit can be specified in time linear in 
the name of the gate (resp. edge). DLOGTIME-uniformity is the strongest no- 
tion of uniformity found generally applicable, and gives nice characterizations of 
ACo, ACo[®] [BIS90]. In this paper whenever we say that a circuit class is uni- 
form we always mean that it is DLOGTIME-uniform. Note that uniform ACo[6] 
is strictly contained in L := logarithmic space. Therefore all our generators are 
in particular explicitly computable in space O(log n). For background on circuit 
complexity we refer the reader to the excellent book by Vollmer [Vol99]. 


' In [MNT90] this negative result is stated for hash functions only, but their techniques 
apply to k-wise independent and ¢-biased generators as well. 
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Table 1. Our main results. 


Generators : {0,1}" — {0,1}™ explicitly computable 
by DLOGTIME-uniform ACo|[@] circuits of size poly(n) 
Seed Tenth Limitations 
n = O(k* log m) k-wise B= Ol Theorem 2 (1) 
Theorem 2 (2) 
) 


n= O(k? log(m) log(k-logm))| k-wise = 
n = O(log m + log 1/e) e-biased je = (2(1/2'°& 8(™)| Theorem 4 (1 
n = O(log mlog 1/e) e-biased - Theorem 4 (2 
n = O(k + log logm + log(1/e))|(e, k)-wise| k = O(log® log(m)),| Omitted due to 
| e = 2(1/2'°% 8°") Ispace restrictions 





1.1 Our Results 


Our main results are summarized in Table 1. We now discuss some of the gen- 
erators in Table 1. 


k-Wise Independent Generators in ACo|@|: k-wise independent generators ex- 
plicitly computable in time poly(n) are known with seed length n = O(k log m) 
[CG89,ABI86] which is optimal up to constant factors (a lower bound of 
(k log m)/2 was proven in [CGH+85]). 

To understand the difficulty of implementing these generators in ACo[6] 
let us discuss a construction from [CG89,ABI86]. For simplicity of exposition, 
assume that m = 2” for some integer h. Let GF(2") be the field of size 2”, then 
for every k, the generator G : {0,1}” — {0,1}” defined as 


G(ao0,@1,...,@k-1)i4 = So a7, where Qo, @1,...,@p—-1,7 € GF(2"), (1) 
j<k 


is a k-wise independent generator? with seed length n = kh = klog m. 

For concreteness, in the following discussion let us fix k at most poly(h), so 
that poly(n) = poly(h). To compute the generator in Equation 1 we must first 
find a representation of GF(2") (e.g., an irreducible polynomial of degree h over 
GF(2)). We do not know how to find a representation of GF(2") in uniform 
ACo|®] circuits of size poly(h) for every given h. And even if one has such 
a representation we do not know in general how to compute field operations 
in uniform AC o[@]. On the other hand, it is known that for every h of the 
form 2-3! (for some l) the polynomial z”? + x”/? + 1 is irreducible over GF(2) 
(see e.g. [VL99], Theorem 1.1.28). Furthermore, it can be shown that using this 
specific representation of GF(2") (for h = 2 - 3!) it is possible to compute field 
operations over GF (2”) in uniform ACo[®] circuits of size poly(h). (Eric Allender 
(personal communication, June 2004) pointed out field multiplication to us. More 
involved techniques seem to give field exponentiation with a slightly growing 


* This generator outputs non-Boolean random variables. To get Boolean random vari- 
ables we can, say, take the least significant bit of G(x). 
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exponent of at most poly(h)). However, these results are bound to a specific 
field representation, and thus are somewhat unsatisfactory. 

In this work we exhibit alternative constructions of k-wise independent gen- 
erators computable in uniform ACo[9]. 

Techniques: Our k-wise independent generators are based on combinatorial 
designs. A combinatorial design is a collection of m subsets S;,..., Smof {1...n} 
with small pairwise intersections. We obtain the i-th output bit of the generator 
by taking the parity of the bits of the seed indexed by S;. Roughly speaking, the 
small intersection size of the sets will guarantee the k-wise independence of the 
variables. For the case k = O(1), we give a new construction of combinatorial 
designs with n = O(log m), whichis computable in ACp[®] (Lemma 3). Following 
an idea from [HRO3] (credited to S. Vadhan), we obtain this construction by 
concatenating Reed-Solomon codes with a design construction from [V1004]. 

To compute our design constructions (specifically the Reed-Solomon code) 
we work over finite fields as well (as does the classic k-wise independent generator 
in Equation 1). The difference is that in our constructions we need finite fields 
of size exponentially smaller, i.e. poly(n) (as opposed to 2”) . Representations 
of such fields can be found by brute force (even inuniform ACg), and results by 
Agrawal et. al. {AAI*01] show how to perform field operations over fields of size 
poly(n) by uniform AC circuits of size poly(n). 

The question rises whether there is a construction of k-wise independent gen- 
erators that does not use finite fields of growing size. We argue (see the remark 
at the end of Section 3) that such a construction is possible using our approach 
based on combinatorial designs, however we only get a generator explicitly com- 
putable in P-uniform ACo{@] (rather than DLOGTIME-uniform). 


€-Biased Generators in ACo|®]: Similar to k-wise independent generators, €- 
biased generators explicitly computable in time poly(n) are known with seed 
length n = O(log m + log1/e) [NN90,AGHP92] which is optimal up to constant 
factors [AGHP92]. 

Alon et. al. [AGHP92] give three simple constructions, but none of them 
seems to be implementable inuniform AC o(®]: they either seem to be inherently 
sequential, or they need large primes, or they need exponentiation over finite 
fields of size 2” with exponent of n bits, which does not seem to be computable 
in uniform ACo[@] circuits of size poly(n). 

Prior to their work, Naor and Naor [NN90] gave another construction. We 
show that their construction (as long as e > 1/2!8'(eg™) for some fixed con- 
stant c) can be implemented inuniform ACo[@] (for e < 1/2! “°s™) we obtain 
seed length O((log m)(log 1/e)) with a slight modification of the construction in 
[NN90]). 

Techniques: The construction of [NN90] goes through a few stages. In partic- 
ular it uses as a component a 7-wise independent generator. By the discussion 
above, we have such a generator inuniform ACo[®]. [NN90] also needs to com- 
pute random walks on an expander graph. We show that, for every fixed c, 
random walks of length log® n on the Margulis expander [Mar73] of size 2”, can 
be computed byuniform ACp circuits of size poly(n) (where the depth of the 
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circuit depends on c). We also argue that for this specific expander, our result 
is tight. Bar-Yossef, Goldreich and Wigderson [ZB YW99] show how to compute 
walks on the same expander but in an online model of computation. Their result 
is incomparable to ours. Ajtai [Ajt93] shows how to compute in uniform ACo 
of size poly(n) walks of length logn on expander graphs of size exponentially 
smaller, 1.e. n. 


Almost k-Wise Independent Generators in ACo|®]|: Using the approach of 
[NN90] that combines k-wise independent and e-biased generators to get al- 
most k-wise independent generators, we can use our constructions to obtain 
an (e, k)-wise independent generator G : {0,1}" — {0,1}™ explicitly com- 
putable by uniform AC o[®] circuits of size poly(n,logm). Its seed length is 
O(k + log log m + log(1/e)) as long as k = O(log® log m) and e = 2(1/2'8" es m) 
(for any fixed constant c > 1). In this range of parameters, this seed length 
matches that of the best known constructions [NN90,AGHP92] up to constant 
factors. Due to space limitations this construction is omitted. 


Organization: In Section 2 we discuss some preliminaries. In Section 3 we show 
the connection between combinatorial designs and k-wise independent gener- 
ators, and we exhibit our construction of combinatorial designs and our con- 
structions of k-wise independent generators inuniform ACo|{@]. In Section 4 we 
give our €-biased generators inuniform ACof]. This includes our results about 
computing walks on expander graphs. 


2 Preliminaries 


We now define k-wise independent and e-biased generators. Denote the set 
{1,...,m} by [m]. For I C [m] and G(x) € {0,1}™ we denote by G(z)|; € 
{0,1}!! the projection of G(x) on the bits specified by J. Recall that @ is the 
parity function, Le. Ð;er(£) := doje, Zi mod 2. 


Definition 1. Let G: {0,1}" — {0,1}™ be a generator. 
— G is (e, k)-wise independent if for every M : {0,1}* — {0,1} and I C |m] 
such that |I| < k: Pie (o,1}*(M(y) = 1] — Preego,ay"(M(G(2)|1) = 1] <e. 
— G is k-wise independent if it is (0,k)-wise independent. 
— G is e-biased if for every I C [m]: | Prze{0,1}r [Dier C(x): = 0) — 3 <e. 


We now define the circuit classes of interest in this paper. ACo is the class 
of constant depth circuits with =,V and A gates, where V and A have un- 
bounded fan-in. ACo[®] is the class of constant depth circuits with ~, V, ^ and 
& gates, where V, A and @ have unbounded fan-in. A family of circuits {Cn} is 
DLOGTIME-uniform, if there is a Turing machine M running in linear time that 
decides the direct connection language of {Cn}, which is the language of tuples 
(t,a,b,n), such that b and a are names of gates in Cn, b is a child of a, and a isof 
type t [BIS90,Vol99]. In this paper uniform always means DLOGTIME-uniform. 
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Definition 2. A generator G : {0,1}" — {0,1} is explicitly computable by 
uniform ACo[@] (resp. ACo) circuits of size g, if there is a uniform ACo|®] 
(resp. ACo) circuit C of size g such that C(z,i) = G(x); for all r € {0,1}" and 
i € {0,1}98™, where G(r); is the i-th output bit of G. 


In [MNT90] they essentially prove the following negative result on the abil- 
ity of ACo circuits to explicitly compute (e, k)-wise independent and e-biased 
generators. 


Theorem 1 ([MNT90]). Fix anyconstant e < 1/2. Let G : {0,1}" — {0,1}™ 
be a generator either (€,2)-wise independent or €-biased. Let C be a circuit of 
size g and depth d such that C(r,i) = G(r); for every r € {0,1}”,7 € {0, 1}98™. 
Then log*~* g > Q(log m). 


In some of our constructions we make use of the following result from 
[AAI*01] about field operations inuniform ACo[@]. We denote by GF(2*) the 
field with 2t elements, and we identify these elements with bit strings of length t. 


Lemma 1 ({AAI‘01], Theorem 3.2 and proof of Theorem 1.1). Let t = 
O(logn). There is a uniform ACo|®] circuit of size poly(n) such that given a 
polynomial p(x) := ye aix? of degree k = poly(n) over GF(2"), and b € 
GF(2°), computes p(b). 


3 k-Wise Independent Generators from Designs 


In this section we present a general approach that gives k-wise independent 
generators from combinatorial designs. First we define combinatorial designs. 
Then we show how to get k-wise independent generators from combinatorial 
designs. We then turn to the problem of efficiently constructing designs with 
good parameters. 


Definition 3. /NW94] A (l,d)-design of size m over a universe of size s is a 
family S = (S1,..., Sm) of subsets of {1,...,8} that satisfies: (1) for every i, 
|S;| = 1, and (2) for every i £ j, |SiN S| < d. 


We now show how to get k-wise independent generators from combinatorial 
designs. 


Lemma 2. Let S = (Si,..., Sm) bea ((k + 1)d,d)-design of size m 
over a universe of size n. Define the generator Gg : {0,1}" — {0,1}” as: 
Gs(r)i := jes, 77. Then Gs is a k-wise independent generator. 


Proof. Fix k output bits 71,...,%,%. We show that Gg(x);, is uniform over {0,1} 
and independent from Gg(zx);,,...,G@s(Z)i,_,- By definition, Gs(zx);, is the par- 
ity of the bits in x indexed by S;,. Since S is a ((k + 1)d,d)-design there is 
e € Si, such that e ¢ Uo<jek Siz Thus the value of Gs(x);, is independent 
from Gg(x)i,,...,G@s(x)i,_,, because its parity includes a bit that is indepen- 
dent from the bits in the parities of Gg(z);,,...,@s(Z)a_,- O 
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3.1 Combinatorial Designs Computable in Uniform ACo/[@] 


We now describe a new construction of combinatorial designs of size m with 
universe size O(log m). In [HRO3] (Section 5.3) it was suggested (following a sug- 
gestion of Salil Vadhan) to combine error-correcting codes with combinatorial 
designs to achieve new designs with good parameters. While their construction 
does not achieve the parameters that we need, we can use their idea to obtain de- 
signs with the desired parameters combining Reed-Solomon codes with a design 
construction from [V1i004]. 


Lemma 3. For every constant c > 1 and large enough m there is a family 
S of (° logm, 2clog m)-designs of size m over a universe of size 50 - clog m. 
Moreover, there is a uniform ACo|®] circuit of size poly(logm), such that given 
i € {0,1}°8™ computes the characteristic vector of Sj. 


Lemma 3 uses as a component a family of designs with exponentially smaller 
parameters computable in ACg [Vio004]. 


Lemma 4 ([Vio04]). For every constant c > 1, and for every large enough n 
there is a family S of (clogn,logn)-designs of size n over a universe of size 
50-c*logn. Moreover, there is a uniform ACp circuit of size poly(n), such that 
given i € {0, i)" computes the characteristic vector of Si. 


Proof (of Lemma 3). First we describe the construction, then we show it is a 
design with the claimed parameters and then we study its complexity. 

Construction: The idea is ‘combining’ a Reed-Solomon code with the designs 
given by Lemma 4. Let n be such that nlogn = clogm. Fix a field F of size n 
(without loss of generality we assume there is such a field). Let h := n/c. Let z 
be a bit string z = ao ...an where each a; has log(n) bits. Define the polynomial 
pz over F as pz := Sa a,x’, Let b1,...,6, be an enumeration of all elements 
of F. 

Now consider a family (D;,..., Dn) of (clog n, log n) designs of size n over a 
universe of size O(c* logn) as guaranteed by Lemma 4. 

Then S = ($},...,Sgh1c¢n) is defined as follows: The characteristic vector of 
S218 Dp, (b,)+**Dp,(b,). Namely it is a concatenation of n characteristic vectors 
of sets from D. 

Analysis: S has 218” = m sets, each of size enlogn = c*logm, and the 
universe size is O(c?n logn) = O(c? logm). We now bound the intersection size. 
Consider z Æ z’. Since p, and p; are polynomials of degree at most h, there are 
at most A distinct b € F such that p,(b) = pz (b). Whenever p,(b) Æ pz (b), we 
have that Dp, œ) and Dp. (b) are distinct sets in the design D, and thus their 
intersection is at most logn. Therefore 


IS N S2| < helogn+(n—h)logn < 2n logn = 2clogm 


Complexity: By Lemma 1, computing p,(b) can be done by uniform ACo[®] 
circuits of size poly(n) = poly(log m), and D can be computed in uniform ACo 
circuts of size poly(n) = poly(logm) by Lemma 4. o 
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The construction above gives k-wise independent generators with k = O(1) 
(see Theorem 2). For k = w(1) we use as a component a celebrated design 
construction by Nisan and Wigderson [NW94] which we now state without proof. 
The complexity of computing this construction follows from Lemma 1. 


Lemma 5 ([NW94]). For every integers £,m such that logm < £ < m, there 
is a (£, log m)-design of size m over a universe of size O(é*). Moreover, there is a 
uniform ACo{@] circuit of size poly(£), such that given i € {0,1} 8" 
the characteristic vector of 9}. 


computes 


We now state our k-wise independent generators. 


Theorem 2. For every large enough m, there is a k-wise independent generator 
G : {0,1}" — {0,1}, that is explicitly computable by uniform ACo[®] circuits 
of size poly(n), where, 


l. k > 2 is a fixed constant, and n = O(k3 logm). Or, 
2. k is any function (of m), and n = O(k? log(m) log(k - log m)). 


Proof. For Item (1) we plug the design construction from Lemma 4 (with c := 
2(k + 1)) into Lemma 2. Item (2) is obtained as follows. Consider a ((k + 
1) log m, log m)-design S of size m over a universe of size O(k? log” m) as guaran- 
teed by Lemma 5. By Lemma 2, Gs is a k-wise independent generator, however 
the seed length of Gg is n = O(k? log? m). To reduce the seed length, suppose we 
have a (k(k + 1) logm)-wise independent generator G’ : {0,1}" — {0,1}". We 
claim that GoG’ : {0,1}” — {0,1}' is still a k-wise independent generator. This 
is because every k output bits of G depend on at most k(k+1) log m output bits of 
G' (x). Since G’ is (k(k +1) log m)-wise independent, these bits will by uniformly 
and independently distributed. Now, using for G’ the generator of [CG89,ABI86] 
(Section 1.1, Equation 1) we have n’ = O(k-(k+1) log(m)- (log k+log(log m)) = 
O(k? log(m) log(k - log m)). Finally, by Lemma 1, G’ is computable by uniform 
ACo[] circuits of size poly(n’). o 


A “Combinatorial” Construction. We note that our approach allows for a con- 
struction of k-wise independent generators (for k = O(1)) that does not use 
finite fields of growing size. This is obtained combining, as in Lemma 3, the 
designs from Lemma 4 with an error correcting code. For the latter, we take an 
expander code [SS96] based on the Margulis expander discussed in Section 4.1. 
The resulting generators match the parameters of Theorem 2 (1), and they are 
computable in P-uniform ACo[®] circuits of size poly(n). 


4 «-Bias in ACo[®] 


In this section we describe our constructions of e-biased generators in uniform 
ACpo|®]. We obtain our constructions by exhibiting uniform ACo[®] implemen- 
tations of the €-biased generator due to Naor and Naor [NN90]. Due to space 
restrictions we do not describe the e-biased generator in [NN90] here. We only 
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point out that it is built combining 7-wise independent generators, 2-wise 1n- 
dependent generators and random walks on expander graphs. Using our k-wise 
independent generators inuniform AC»o[®] (Theorem 2), all that is left to do is 
computing walks on expander graphs in uniform ACo[]. We now formally state 
what random walks we need to get ¢-biased generators. 

A family of graphs {Gy }n is a family of d-regular expander graphs if there is 
a constant A < d such that for every N the graph Gy has N nodes, is d-regular, 
and the absolute value of the second eigenvalue of its adjacency matrix is at most 
A. To get an e-biased generator we need random walks of length | = O(log 1/e) 
on expander graphs of size 2°‘). We show in the next section that for every 
fixed c we can compute walks of length log* n on certain expander graphs of size 
2” inuniform ACp circuits of size poly(n). 


Theorem 3. There is a family {Gy}w_ of 8-regular expander graphs such that 
for every c there is a uniform ACo circuit of size poly(n) that, given v E€ Gon 
and a path w of length log? n, computes the node v' € Gon reached starting from 
v and walking according to w. 


Using the expander walks in Theorem 3 we get, for e > 1/2! “°e™) an 
€-biased generator with seed length optimal up to constant factors [AGHP92]. 
For smaller €, we replace the random walk on the expander with random (inde- 
pendent) nodes in the graph, and obtain a generator with larger seed. 


Theorem 4. For every large enough m, there is an €-biased generator Ge : 
{0,1}" — {0,1}™, explicitly computable by uniform ACo[®| circuits of size 
poly(n), where 


1. n= O(log m + log(1/e)) and e = 1/28 008m), for any constant c > 0. Or, 
2. n = O((log m) log(1/e)) and € = €(m) is arbitrary. 


4.1 Expander Walks in ACo 


In this section we prove Theorem 3, i.e. we show that there is an expander graph 
of size N = 2” where random walks of length l = O(log? n) can be computed by 
uniform ACg circuits of size poly(n) (for every fixed constant c; the depth of the 
circuit depends on c). We use an expander construction due to Margulis [Mar73] 
(the needed expansion property was proved later in [GG81,JM87]), which we 
now recall. 

Let m := yN (we assume without loss of generality that m is a power of 
2). The vertex set of Gy is Zm X Zm, where Zm is the ring of the integers 
modulo m. Each vertex v is a pair v = (x,y) where x,y € Zm. For matrices 
Tı, Tọ and vectors b,,b2 defined below, each vertex v € Gy is connected to 
Tiv, Tiv + b1, Tov, Tov + ba and the four inverses of these operations. 


01 
10 1 O\. 
(G 4) bı = o and bg := (2) is a family of 8-regular expander graphs (the 


absolute value of the second eigenvalue of the adjacency matrix is 5/2 < 8). 


Theorem 5. [Mar73,GG81,JM87] The family {Gy }n with Ti := (a a Tə := 
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We now prove Theorem 3, with the family {Gy }n of expander graphs from 
Theorem 5. That is, we show that walks of length log n on Gan can be computed 
in uniform AC circuits of size poly(n). We note that this result is tight, i.e. there 
isno ACo[@] circuit (uniform or not) of size poly(n) that computes random walks 
of length log’ n on Gon. This is because computing >>, z; given x € {0,1}! 
(which cannot be done in ACo[@] for | = log’ n) can be ACo reduced to 
the problem of computing random walks of length O(l) over Gon. (Proof sketch: 
Given z, replace ‘1’ in x with a step along the edge associated with T}, replace ‘0’ 
in x with a self-loop. Call x’ the string thus obtained. Now start at vertex (0, 1), 
and compute a walk according to z’. It is easy to see that the first coordinate of 
the ending node is `; x;). However this negative result relies on the particular 
expander graph and on its representation. We do not know if ACo[®] (or ACo) 
circuits of size poly(n) can compute random walks of length log?) 
expander on w(n) vertices. 

We now turn to the proof of Theorem 3. By definition of Gy, there are 
8 matrices 7;,.. s Ty with constant size entries and 8 vectors by. ; bs with 
constant size Sitries such that the set of neighbors of a vertex v are {Tv + by : 
i < 8}. Thus computing the random walk translates to computing 


n on some 


v = Ail. 3a (A3(Ao(Aiv -+ a1) + a2) + a3) ui y + a; 
where forevery i, A; € {T, :i < 8} and a; € {b; : i <8}. We write this as 
v = Av+A’ where A := A er AoA and A’ = Al saa Asai +Áı TE Azaz +. .. +a]. 


So we are left with the following tasks: computing the matrixes A, A’ and then 
computing Av + A’. We now show how to solve these problems inuniform AC. 
First note that since the matrixes A,’s have constant size entries, the matrixes 
A and A’ have entries of size at most O(l) = log n bits for some c. 

To compute Av + A’ given A, A’ and v we use the facts that (1) sum of two 
n-bit integers is in ACp and (2) multiplication of a n-bit integer by a log® n-bit 
integer is in ACo. For (1) see e.g. [Vol99], Theorem 1.20, for (2) see e.g. [Vol99], 
Theorem 1.21 (this latter theorem shows multiplication of a n-bit integer by 
a log n-bit integer. The same techniques give (2)). These circuits can be easily 
shown to be uniform. 

We now show how to compute A, the same techniques give A’. 


Lemma 6. For every fixed constant c there is a uniform ACo circuit of size 
poly(n) that, given | = log*n, 2 x 2 matrixes A,,...,A, with constant size 
entries, computes A= | |;<, A 


Proof. Instead of proving the lemma directly, it is convenient to show that the 
product of n (as opposed to l) given matrices Aj,...,An with constant size 
entries can be computed in space O(logn), and then appeal to the following 
lemma (that can be obtained combining results in [Nep70] and in [BIS90], details 
omitted). 
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Lemma 7 ([Nep70] + [BIS90]). Let L be a language computable in logarith- 
mic space. Then for every constant c > 1 there is a uniform ACo circuit of size 
poly(n) that, given x of size logn, correctly decides whether x is in L. 


Suppose we are given n matrices A;,..., A, with constant size entries. The 
idea is to first compute ] |;<„ A; in Chinese Remainder Representation (CRR) 
and then convert the CRR to binary (using a result by Chiu, Davida and Litow 
[CDLO01]). More formally, note that every entry of [ [;<p Ai, for every k < n, will 
be at most d”, for some constant d. By the Chinese Remainder Theorem each 
number x < d” is uniquely determined by its residues modulo poly(n) primes, 
each of length O(log n). The Prime Number Theorem guarantees that there will 
be more than enough primes of that length. We refer to such a representation of 
a number 2 (i.e. as a list of residues modulo primes) as the CRR of x. Note that 
to find the primes for the CRR we search among the integers of size O(log n). 
This clearly can be done in logarithmic space. 

We compute the product of the n matrices modulo one prime at a time, 
reusing the same space for different primes. To compute the product modulo 
one prime first note that transforming a matrix in CRR is easy because the 
matrixes have constant size entries. Each matrix multiplication is a simple sum 
(because the matrices have constant size entries) and therefore can be computed 
in logarithmic space. The machine operates in space O(log n) because it only 
needs to store a constant number of residues modulo a prime of length O(log n). 

All that is left to do is to convert the product matrix from CRR to binary, 
and this can be done in space O(log n) by a result in [CDLO1]. o 
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Abstract. We consider distribution-free property-testing of graph con- 
nectivity. In this setting of property testing, the distance between func- 
tions is measured with respect to a fixed but unknown distribution D on 
the domain, and the testing algorithms have an oracle access to random 
sampling from the domain according to this distribution D. This no- 
tion of distribution-free testing was previously defined, and testers were 
shown for very few properties. However, no distribution-free property 
testing algorithm was known for any graph property. 

We present the first distribution-free testing algorithms for one of the 
central properties in this area - graph connectivity (specifically, the prob- 
lem is mainly interesting in the case of sparse graphs). We introduce three 
testing models for sparse graphs: (1) a model for bounded-degree graphs, 
(2) a model for graphs with a bound on the total number of edges (both 
models were already considered in the context of uniform distribution 
testing), and (3) a model which is a combination of the two previous 
testing models; 1.e., bounded-degree graphs with a bound on the total 
number of edges. We prove that connectivity can be tested in each of 
these testing models, in a distribution-free manner, using a number of 
queries independent of the size of the graph. This is done by providing 
a new analysis to previously known connectivity testers (from “‘stan- 
dard”, uniform distribution property-testing) and by introducing some 
new testers. 








1 Introduction 


The classical notion of decision problems requires an algorithm to distinguish 
objects having some property P from those objects which do not have the prop- 
erty. Property testing is a relaxation of decision problems, where algorithms are 
only required to distinguish objects having the property P from those which are 
at least “e-far” from every such object. The main goal of property testing is to 
avoid “reading” the whole object (which requires complexity at least linear in 
the size of its representation); 1.e., to make the decision by reading a small (pos- 
sibly, selected at random) fraction of the input (e.g., a fraction of size polynomial 
in 1/e and poly-logarithmic in the size of the representation) and still having a 
good (say, at least 2/3) probability of success. 

The notion of property testing was introduced by Rubinfeld and Sudan [19] 
and since then attracted a considerable amount of attention. Property testing 
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algorithms (or testers) were introduced for problems in graph theory (e.g. [1, 9, 
11, 17]), monotonicity testing (e.g. [5, 8, 14]) and other properties (e.g. [2,4, 16]; 
the reader is referred to surveys by Ron [18], Goldreich [7], and Fischer [6] for a 
presentation of some of this work, including some connections between property 
testing and other areas). In these papers, the distance between an object and 
the property is measured by the fraction of the domain that has to be modified 
in order for the object to obtain the property. For example, when dealing with 
graphs, this distance is measured by the number of edges that should be added 
or removed so that the given graph posses the property. 

However, it is natural to consider situations where not all edges are equiv- 
alent. That is, removal or addition of some edges may be more expensive than 
others, depending on various factors of the problem. We are interested in taking 
these differences between the edges into consideration when measuring the dis- 
tance between a given graph and the property. In other words, we wish to put 
different weights (probabilities) on different elements of the domain. This notion 
of testing, termed distribution-free testing, has been previously introduced by [9] 
and some negative results for distribution-free testing of certain graph proper- 
ties were proved (see below); In [13] the authors presented the first (non-trivial) 
distribution-free testers for two properties: monotonicity and low-degree poly- 
nomials. This raises the question whether distribution-free testers can be found 
for problems in other areas, and specifically graph properties, where the concept 
of non-uniform weights for the points of the domain seems natural. 

In this paper, we study distribution-free testing of graph properties, and 
more specifically the connectivity property. In this case, we wish to distinguish, 
with high probability, connected graphs from those that are far from any such 
graph. The distance between two graphs is measured with respect to some fixed 
but unknown distribution D. Testing of graph properties in general, and con- 
nectivity in particular, are among the most central problems studied in the field 
of property testing (with respect to the uniform distribution). Below we survey 
some of the relevant previous work. 


Graph Representations and Previous Results: In the study of graph properties 
(e.g. [1, 3, 9-12, 15, 17]), most previous work distinguish between two basic mod- 
els - one is more suitable for dense graphs while the other is more suitable for 
sparse graphs. In the first model, a graph is represented by its adjacency matrix 
or by afunction f : V x V — {0,1}, specifying forevery u,v € V whether there 
is an edge between u and v. The distance between two graphs in this model, 
is measured by the fraction of entries in the matrix that should be altered in 
order to transform one graph to the other. This model was first considered (in 
the context of property-testing) by Goldreich, Goldwasser and Ron in [9], who 
proved that a variety of graph-partition problems can be tested with query com- 
plexity poly( 1) (which is independent of the size of the graph). Testing in this 
graph model was further studied, for example, in [1], where a characterization 
is given of the first order properties that are testable using a number of queries 
independent of n. 
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Though representation as an adjacency matrix seems very natural, in the 
context of testing it is only relevant when dealing with dense graphs; that is, 
graphs whose number of edges is 2(n?). This is because every two sparse graphs 
(that contain o(n”) edges) are close in the adjacency matrix representation. 
Hence, another model is needed to deal with sparse graphs. Property-testing 
for sparse graphs was first considered by Goldreich and Ron [11], who defined 
the following representation for bounded-degree graphs. Let G = (V, E) be an 
undirected graph with bounded-degree d (i.e., every vertex in V has at most d 
neighbors) and assume that, for every vertex v € V, there is an ordering among 
the neighbors of v. The graph G is viewed as afunction f : V x {1,...d} > VU 
{L} where f(v,i) is set to be u if the i” neighbor of v is u, and is set to be Lif v 
has less than i neighbors. Notice that, since the ordering of the neighbors of each 
vertex is arbitrary, a graph G may have more than one function representing it. In 
this representation, the distance between two graphs is measured by the fraction 
of f values that have to be changed in order to transform one function into the 
other. [11] shows, for example, that in this graph representation, connectivity, 
k-edge connectivity, k-vertex connectivity and cycle freeness are testable using a 
constant (i.e., a function of e7}, but independent of n and d} number of queries, 
and that planarity is testable using O(d* . e7t) queries. Properties of graphs 
represented as incidence lists of length d have also been studied in [3] and [10]. 

A different representation was presented by Parnas and Ron [17] for testing 
properties of all sparse graphs, rather than only the bounded-degree ones. In 
this case, the graphs are represented by incidence lists of varying length. For 
each vertex v the tester can query both the degree of v and the i'th neighbor 
of v. The distance between two graphs is then measured by the fraction of edge 
modifications necessary to obtain the property, defined with respect to an upper 
bound m on the number of edges in the graph. They present a testing algorithm 
for graph diameters, in this representation. In addition, they explained how many 
of the results proved in [11] for the bounded-degree case, can be transformed to 
this representation. Among these properties are the testing algorithms for k- 
connectivity. This graph representation was further studied in [15], where the 
testing model is adjusted to deal with dense graphs as well, by allowing the tester 
to query whether there exists an edge in the graph between two given vertices. 
The authors study testing of graph bipartiteness in this model. 


Connectivity Testing: Connectivity is a central property in sparse graphs; as 
mentioned above, it has been previously studied in the context of testing with 
respect to the uniform distribution. However, although extending the testing 
models for graph problems to the distribution-free setting seems natural, the 
problem of connectivity testing in the distribution-free setting has not been dealt 
with before. Moreover, as mentioned, the only known results for distribution-free 
testing of graph properties in general are impossibility results for graphs repre- 
sented by adjacency matrix [9]. They proved that it is impossible to test a variety 
of partition problems (for which they showed testers with respect to the uniform 
distribution) in a distribution-free manner. The generalization of the adjacency 
matrix model to a distribution-free one, is straightforward; on the other hand, 
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every graph is close to be connected in this representation with respect to any 
distribution’, hence testing connectivity is trivial. However, unlike the dense 
case, when dealing with representations for sparse graphs, their generalization 
to the distribution-free case is not so straightforward, and the testing is not 
obvious. 


Our Contributions. We consider three representations of sparse graphs in the 
distribution-free setting. These representations are generalizations of the ones 
presented previously for sparse graphs in the context of uniform distribution 
testing. We show that in each of these representations, connectivity can be tested 
in a distribution-free manner, using a number of queries independent of the size 
of the graph. 


— The first representation is for bounded-degree graphs [11]. We show that it is 
possible to test graph connectivity, in this representation, in a distribution- 
free manner, using a number of queries which is polynomial in L, The test 
is similar to the one presented in [11] for the uniform setting, however the 
analysis required for the distribution-free case is different (Section 3). 

— The second representation is for graphs with a bound m on the total number 
of edges [17]. We show that in this representation, it is possible to test 
connectivity, in a distribution-free manner, whenever m > n(1 +c) for some 
constant c using a poly(+) number of queries (Subsection 4.1) . 

— We then combine the two previous testing models, and deal with bounded- 
degree graphs with at most m edges. This model was not studied before 
even in the uniform setting. Hence, we first present a uniform distribution 
tester for this model for every m; then, using the distribution-free tester for 
the previous model, we prove that there exists a distribution-free tester for 
graph connectivity in this model whenever m > n(1 + ¢), for some constant 
c (Subsection4.2). 


2 Definitions 


In this section, we define the notions of being e-far from a property P with 
respect to a given distribution D, and of distribution-free testing. Denote the 
range of functions in question by A, and denote by [n] the set {1,..., 7}. 


Definition 1. Let D be a distribution over a set X. The D-distance between 
functions f,g:X — A isdistp(f,g) = Pre~p{f(z) Oat) |: 
The D-distance of a function f froma property P (i.e., the class of functions 
satisfying the property P) is distp(f,P) = min,ep distp(f,g). We say that f is 
(e, D)-far from a property P if distp(f,P) > e. 

When the distribution in question is the uniform distribution over 1, we 


either use U instead of D or (if clear from the context) we omit any reference to 
the distribution. 


' Under any distribution measure on V x V, the total probability of the path with 
minimal probability is at most L, 
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Next, we define the notion of distribution-free tester for a property P. Note 
that the type of queries possible for the tester is determined by the model (as is 
the case in the literature on testing with respect to the uniform distribution). 


Definition 2. A distribution-free tester for aproperty P isaprobabilistic oracle 
machine M, which is given a distance parameter € > 0, and an oracle access to 
an arbitrary function f : X — A and to sampling of a fixed but unknown 
distribution D over X, and satisfies the following two conditions: 

1. Iff satisfies P, then Pr{Mf? = Accept} = 1. 

2. Iff is (e, D)-far from P, then Pr{Mf = Accept} < 3. 

Note that a more general definition of testers that allows two-sided errors is not 
needed here; all our testers, like many previously-known testers, have one-sided 
error and always accept any function that satisfies the property P in question. 

The definition of a uniform distribution tester for a property P can be derived 
from the above definition by omitting the sampling oracle (since the tester can 
sample in the uniform distribution by itself) and by measuring the distance with 
respect to the uniform distribution. 

Finally, notice that since the distribution D in question is arbitrary, it is 
possible that there are two distinct functions f and g such that distp(f,g) = 0. 
Moreover, it is possible that f ¢ P and g € P. Since the notion of testing is 
meant to be a relaxation of the notion of decision problems, it is required that the 
algorithm accepts functions that satisfy P, but may reject functions that have 
distance 0 from P and do not satisfy P. In addition, note that the algorithm is 
allowed to query the value of the input function also in points with probability 
O (which is also the case with membership queries in learning theory)*. This 
definition of distribution-free testing was introduced in [9, Definition 2.1]. 


3 Connectivity Testing of Bounded-Degree Graphs 


The representation for bounded-degree graphs which is the subject of this section 
is a generalization of [11] to deal with arbitrary probability distributions. First, 
we generalize the notion of afunction fg : V x [d] > V U {L} that represents 
a degree d (undirected) graph G. Since the d outgoing edges of a vertex v may 
have different probabilities, it is essential to allow fg¢(v,i) to equal L even if 
fa(v,i+1) # L (ie., we cannot assume that a node of out-degree d’ < d uses 
the first d’ entries in its incidence list). 


Definition 3. A function fg : V x[d] => VU{1} represents a graph G = (V, E), 
if the following holds: 

(a) for every edge (u,v) € E there exist unique i, and ty such that falu, iu) =v 
and fg(v,ty) =u, and 

(b) for every vertex v € V andi € |d], if there exists no neighbor u of v such 
that fa(v,i) = u, then fa(v,i) = L. In this case, we say that the pair (v,i) is 
free in fq. 








* It is not known whether membership queries are essential in general for testing even 
in the uniform case (see [18]); such a result is known only for specific problems such 
as monotonicity testing (see [5]). 
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Denote by P% the class of functions f that represent connected graphs with 
bounded-degree d. Given a probability measure D : V x [d] — [0,1] and a 
function fg that represents a graph G, we are interested in the D-distance of fg 
from P?. Hence, we examine possible ways to transform fg into a function fa 
that represents a connected graph G” with degree d (i.e., fg: € Pt). To do so, 
we have to connect all the connected components of G; that is, if the connected 
components of G are C},...,C;, then we wish to add an edge between C; and 
Ci41 while, at the same time, preventing the graph’s degree from exceeding d, 
and keeping the connectivity of each of the connected components. In addition, 
we want the total probability of the modified (1.e., added and removed) edges to 
remain small. Therefore, we look, in any connected component C; of G, either 
for an edge that is unnecessary to the connectivity, or for two distinct free pairs 
(v, iv) and (u,2,). In other words, we find a list of possible connecting points 
between the connected components of G. We define more accurately the notion 
of such a list. For this purpose, we need the following definition. 


Definition 4. Let G = (V, E) be a graph, let e = (u,v) € E be an edge, and let 
C be a connected component of G containing e. We say thate is redundant, if 
removing e from E does not affect the connectivity of C. 


The following definition formally states the idea of a list of possible connec- 
tions between the connected components of a graph. The m’th edge in this list, 
(Um; Um), Will connect the components Cm and Cm+1. 


Definition 5. Let fg be a function that represents a graph G with bounded- 
degree d, and let Cy,...,Cx be the connected components of G. We say that a 
list of k quad-tuples L = ((uy,%1),(v1,71)),---, (uk; ik), (Vk, Jk)) connects fa if 
the following holds, for every m € fk]: 

l]. Um E Cm and Um E Cm41 (in case m = k, then m + 1 refers to 1). 

2; (idm) £ (Uitte): 

3. one of the following holds: (a) (um+1,im41) and (Vm, jm) are free in fa; or 
(b) (Um+1,Um) is redundant in Cm+1, and fg(Um+41,im4i1) = Um; falm, jm) = 
Um+1 a 


Indeed, given a list L that connects fg, we can construct from fg afunction 
fo, that represents a connected degree d graph G’, by setting fa (Uum, im) = 
Um and for(Um,jm) = Um (notice that the graph G” is obtained from G by 
removing at most one redundant edge from each connected component, thereby 
not damaging the component’s connectivity). 

Let fg be a function that represents a graph G. For every edge (u,v) € 
E, define the D-probability of the edge (u,v) to be the total probability of its 
endpoints with respect to D (it can be seen as the cost of a change in the 
edge (u,v), since a change in the edge (u,v) requires a change in the incidence 
lists of both u and v); equivalently, let i, and i, be such that fa(u, iu) = 


> The above definition actually describes a cycle connecting all the connected compo- 
nents of the graph. It is possible to define a path from C to Ck; however, in such 
a case, not all the quad-tuples are symmetric, causing the definition to be slightly 
more complicated. 
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v and fc(v,iy) = u, then the D-probability of the edge (u,v) is D(u,v) = 
D(u, iu) + D(v,i,). In addition, define the D-probability of a vertex v € V 
to be D(v) = ŽicjaD(v,i) and the D-probability of a list L to be D(L) = 
dim(D(um, tm) + D(Um; jm)). 


Observation 1: If a list L connects fg, then the D-distance of fg from P? is 
at most D(L). 


We present an O(e~) distribution-free tester for connectivity of bounded- 
degree graphs that, given access to random sampling of V x [d] according to the 
distribution D and to membership queries of a function fg, distinguishes, with 
probability at least 2 between the case that fg represents a connected graph 
with bounded-degree d (i.e., fo € PÊ), and the case that fa is (e, D)-far from 
P2. For lack of space, we deal here only with the case where d is at least 3. An 
algorithm for the special case d = 2 can be constructed using similar arguments 
and will appear in the full version of the paper. 

The tester is similar to the one presented in [11] for the uniform distribution, 
in the sense that it also looks for small connected components in G. However, 
while the original analysis is based on the fact that the number of small connected 
components in a graph G’ which is far from being connected, is big, this claim 
no longer holds when dealing with arbitrary distributions’. Hence, a whole new 
analysis is required for the distribution-free case. A natural generalization of the 
tester for the uniform case may seem to be seeking for connected components 
where the total probability of their vertices is small (note that, there is no 
correlation between the number of vertices in a connected component and their 
total probability). However, there are some drawbacks to this approach. First of 
all, in the distribution-free setting we have no knowledge of the actual probability 
D of the sampled points, and are only allowed to sample the domain according to 
D; hence, we are only able to estimate their probability. In addition, the size of 
such components may be very large, therefore finding out whether two vertices 
lie in the same component may not be possible in time independent of the size 
of the graph. Thus, a different generalization is required. 


Algorithm - connectivity(e, d) 
Repeat $ times: 
— Choose, using the sampling oracle, (v,i) ~ D. 
— Perform BFS starting from v until z vertices have been reached, or no 


new vertex can be reached. If the search was ended since no new vertex 
can be reached, return FAIL®. 


return PASS 


* Consider for example a graph G that consists of two connected components, one of 
size |V| — 1 and the other containing a single vertex vo, and the distribution D is 
set to be D(vo,i) = 4 for every 1 <i < d. In this case, G is (5, D)-far from being 
connected, while it contains only one small connected component 

5 Assume |V| > 28. If |V| < 28, then it is possible to decide whether the graph is 
connected in query complexity O(e7*). 
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To prove the correctness of this algorithm, we use the following notation. Let 
fa be a function that represents a graph G, and let C be a connected component 
of G; denote by nc the number of vertices in C and define wc = $ ec D(v) 
(1.e., the total probability of all the vertices in C). 

The next lemma states that if fg is (€, D)-far from P%, then the total weight 
of the small connected components in the graph G is at least į. Denote by 
Sp(fa) the total probability of connected components in G whose size is smaller 


than 3s. Lex Sp( fa) = l Gnod wC. 
Lemma 1. Let fg be a function (e, D)-far from P*. Then, Sp(fa) > A 


To prove the lemma, we show that if Sp( fe) < §, then it is possible to construct 
a list L that connects fg, such that D(L) < e. Then, by Observation 1, we 
deduce that distp(fg, P$) < e. The full proof is omitted and will appear in the 
full version of the paper. 

We can now prove the correctness of the algorithm connectivity. 


Theorem 1. Algorithm connectivity(e,d) is a distribution-free tester for con- 
nectivity of bounded-degree graphs with degree d > 3; its query complexity is 
O(e-). 


Proof. By the definition of a function representing a graph, in every stage of 
the BFS the algorithm looks for the next neighbor of the current vertex, which 
causes the BFS step to cost O(d) queries (in the worst case). Since there are 
O(<) BFS steps, the query complexity is as required. By the definition of the 
algorithm, if fa € P%, then it is accepted by the algorithm with probability 1. Let 
fa be (e, D)-far from Pt. By Lemma 1, Sp(fe) > á. Hence, the probability of 
the algorithm to randomly pick a pair (v,i) such that v € C and C is a connected 


component of size less than 28, is at least 5. Therefore, the probability that the 


2 
algorithm fails to find a small component is at most (1 — s) <i <i. m 


4 Alternative Testing Models for Sparse Graphs 


In this section, we consider additional testing models for sparse graphs, as dis- 
cussed in the Introduction. For each of these testing models, we define the class 
of functions that represent graphs, the queries that the tester is allowed to ask, 
and prove the existence of a distribution-free tester in that model. Section 4.1 
deals with graphs with a bounded number of edges. Then, Section 4.2 deals with 
bounded-degree graphs with a bound also on the total number of edges. 


4.1 Graphs with a Bound on the Number of Edges 


The testing model described in this section is a generalization of the model 
presented in [17] for testing with respect to the uniform distribution: There is no 
upper bound on the degree, only a bound m on the total number of edges in the 
graph; as in the uniform model, the tester can query the degree of a given vertex 
v or the i'th neighbor of v. As explained in [17], connectivity can be tested with 
respect to the uniform distribution. 
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To generalize this model to the distribution-free setting, we view a graph 
G = (V, E) as afunction f : [m] — (Y) U{L}, indicating for each i € [m] the 
i'th entry in the edge list if such an edge exists, or L if that entry is empty. 
As before, a graph G can have more than one representation, due to different 
orderings of the edge list. Denote by Pm the class of functions that represent 
connected graphs with at most m edges. 

Note that, the differences between the weights of the entries in the list are 
more significant when removing edges (in order to make room for new edges) 
and less relevant when talking about edges additions (where one can choose the 
cheapest entry). 

The next lemma, shows that if m > (1+ c)n for some constant c, then it is 
possible to use any tester known in this model for the uniform distribution, to 
construct a distribution-free tester. 


Lemma 2. Let m > (1+c)n, for some constant c. Then, there exists a distri- 
bution-free tester for Pm with query complexity of O(e7?). 


To prove the lemma, we show that if fg is e-far from Pm with respect to any 
distribution D, then it is also g(e)-far from Pm with respect to the uniform 
distribution, when g(e€) is a linear function in e. (proof omitted.) 


4.2 Graphs with a Bound on the Number of Edges 
and on the Degree 


Next, we consider a combination of the two previous testing models. One may 
think of graphs where there are at most m edges and the degree of each vertex 
is at most d. This model can be seen as an intermediate model between the 
above two models, since it contains all bounded-degree graphs and is contained 
in the set of all graphs with a bound on the total number of edges. As in the 
previous model, graphs are represented by functions f : [m] — (Y) U{L}, and 
the distribution is defined over the list of edges, i.e. the set [m]. However, in this 
case we are only interested in functions that represent degree-d graphs with at 
most m edges. Denote the class of these functions by P%,. As in the previous 
model, the tester is able to query the i'th neighbor of a given vertex v. 

Though we believe that this is a very natural testing model for sparse graphs, 
it was not considered before even in the uniform setting. We prove existence 
of both uniform and distribution-free testers for P4, using similar approaches. 
When dealing with the uniform distribution, the distance between two functions 
is measured as in [17]; that is, by the fraction of edge modifications necessary to 
transform the graph into a connected one, measured with respect to m. 


Lemma 3. There exists a uniform tester for P$, with query complexity poly(+). 


To prove the above lemma, we show that if a function fg is e-close to Pm, 
then it is 3e-close to P, (note that both testing models use the same graph 
representation). It follows, that if fe is e-far from P$, it is s-far from Pm. 
Hence, the existence of a uniform tester for P$, follows from the existence of 
such a tester for Pm [17]. The full details are omitted for lack of space. 

We now turn to the distribution-free setting. Assume that d > 3; the proof 
for d = 2is similar. 
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Theorem 2. Let m > (1+c)n, for some constant c. There exists a distribution- 
free tester for PÉ, with query complexity poly(+, d). 

To construct the distribution-free tester, we want to use the tester presented for 
bounded-degree graphs in Section 3. Hence, we first need a way to transform 
graphs in our representation to the bounded-degree representation. We describe 
how, given a function fe : [m] > (Y) U{1} that represents a graph G with 
degree at most d, we transform it to a function T(fg) : V x [d] — VUH} 
that represents the same graph G. The transformation is as follows: for every 
v € V, assume an ordering on the edges outgoing from v (i.e., the same ordering 
according to which the tester’s queries are answered). For every i € |d], define 
the value of T(fg)(v,1) = u if u is the i'th neighbor of v and L if v has less than 
i neighbors. Note that necessarily if T(f)(v,i) = L, then T(f)(v,i +1) = L. 

Note that, given access to queries for the function fg, one can simulate 
queries for the function T'( fg) (this is due to the choice of the same ordering on 
the outgoing edges of a vertex v). 

However, in order to use the distribution-free tester for bounded-degree 
graphs, we need to also find a way to translate distribution measures from our 
model to the bounded-degree model. 

Given the functions fg and T'(fe), we describe a transformation from the 
distribution D defined over [m] to a distribution T(D, fg) defined over V x [d]. 
Define W to be the total probability of all non-empty entries in [m] under the 
distribution D. That is, W = 7; 54, D(i). The distribution T(D, fe) is now 
defined as follows. For every v € V and i € [d]: 

o If (fe)(v,7) = u and (v, u) is the 7’th edge in the list, define T(D, fg)(v,1) = 
D(j)/2W. If W = 0, then T(D, fa) is defined to be the uniform distribution 
over all non-empty entries in |m]. 

e Otherwise, define T(D, fav, i) = 0. 

Note that, sampling of V x [d] with respect to the distribution T(D, fg) using 
sampling of [m] according to the distribution D is not possible. This is due to 
the possibility to sample empty entries in [m], that cannot be translated to a 
pair (v,i) in V x [d]. 


Sampling of T(D, fa): Set ¢ = 8. 

e Sample, at most Be log + times, the distribution D until the sampled value 7 is 
such that f(i) Æ L. 

e If all ne log + samples were of empty entries, return FAIL. 

e Otherwise, let i be the sampled non-empty entry, and assume f(z) = (v, u). 
Find the location of u in v’s neighbors list, denote it by i,. Similarly, find v’s 
place in w’s neighbors list, denoted by ia. Then, with probability return (v, iv), 
otherwise return (u, tiu). 


Each sampling of T(D, fa), using the above sampling procedure, requires 
poly(, d) queries. In addition, if the sampling procedure indeed returns a value 
(v,i), then (v,i) ~ T(D, fa). However, there is also the possibility that the 
sampling procedure returns FAIL, which does not happen in direct sampling 
of T(D, fa). Let A be a connectivity distribution-free testing algorithm in the 
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bounded-degree model. Denote by A’ the algorithm obtained from A by adding 
the following rule: if any sampling query of A, during its run on a function f, 
was answered FAIL, then A accepts f. We will show that we can still use the 
tester after this modification for our construction. 

Assume for now, that we actually have an ability to sample T(D, fa). One 
may wonder whether the distribution-free tester for P, that was presented in 
Section 3, can be used as a tester for P4 as well. It is easy to see, however, that in 
some cases the function T( fg) is very close to P? with respect to T(D, fc), while 
fo is far from P£, with respect to D °. Therefore, a different test is required. 
The distribution-free tester for P%, is as follows: 


Algorithm Connectivitym (€, d): 

Repeat twice: 
Run connectivity’(e,d) (obtained from the algorithm presented in 
Section 3) on T( fc) using the sampling procedure for T(D, fc) with 
distance 5. 

Run the distribution-free tester for Pm on fa and D with distance 5. 


return PASS 


Theorem 2 now follows immediately from the correctness of the above tester. 
This, in turn, is based on Lemma 4 and Lemma 5 below, and the existence of 
the distribution-free testers for Pm (Section 4.1). As in Section 3, we denote 
by Sp(f) the total weight of the small connected components in G. That is, 
the total weight of connected components of size less than 28 Lemma 4 below 
shows that if the total weight of small connected components in G, measured 
with respect to D, is large then, with high probability (over the possible runs 
of the tester), the function fg fails to pass the first stage of the above tester. 
Lemma 5 then shows that if the total weight of small connected components in 
G is small, and the function is far from P2, then with high probability, over the 
possible runs of the tester, it fails to pass the second stage of the above tester. 


Lemma 4. Let fc : [m] — (Y) U{L} be a function and D a distribution over 
Im]. IfSp(fe) > $, then the probability that fg passes the first stage of the 
tester is at most §. 


The proof of this Lemma uses the following observation, which is based on the 
analysis of the tester for P? (Section 3). 


Observation 2: Let fg be a function in the bounded-degree model. If Sp(fe) > 
5, then the probability that f passes the distribution-free test presented in Sec- 
tion 3, with respect to a distribution D over V x |d] with distance parameter e, 
is at most 5. 


° Letm = nd/4, let D be the uniform distribution over[m], and consider the following 
graph G on n nodes. G consists of n/2 isolated vertices and a connected component 
of size n/2 that contains nd/4 edges. It is easy to see that distp(fc,P%,) > 1/d, 
while distr(p so (T (fa), P?) = 4/nd. 
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Lemma 5. Let f : [m] > (YJ U{L} be a function and D a distribution over 
Mae is 5-close to Pm with respect to D, and Sp(fa) < $, then f is €-close 
to På , 
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Abstract. A k-uniform hypergraph G of size n is said to be é-far from 
having an independent set of size pn if one must remove at least en” edges 
of G in order for the remaining hypergraph to have an independent set of 
size pn. In this work, we present a natural property testing algorithm that 
distinguishes between hypergraphs which have an independent set of size 
> pn and hypergraphs which are £-far from having an independent set of 
size pn. Our algorithm is natural in the sense that we sample ~ o(k) s 
random vertices of G, and according to the independence number of the 
hypergraph induced by this sample, we distinguish between the two cases 
above. Here c(k) depends on k alone (e.g. the sample size is independent 
of n). To the best of our knowledge, property testing of the independence 
number of hypergraphs has not been addressed in the past. 























1 Introduction 


A k-uniform hypergraph is a hypergraph G = (V, E) in which each (hyper) edge 
is of size exactly k. An independent set J in G, is a subset of vertices that do not 
include any edges (i.e. there does not exist an edge {v1,..., Yk} E€ E for which 
v; € I forall i € {1,...,k}). The size of the maximum independent set in G is 
denoted by a(G), and referred to as the independence number of G. Consider a 
k-uniform hypergraph G of (vertex) size n, with a maximum independent set 7 
of size a(G). Let H be a random subgraph of G of size s (i.e. H is the subgraph 
induced by a random subset of vertices of G of size s). In this work we study 
the independence number of H. 

For example, if it is the case that a(G) > pn forsomeparameter p, it is 
not hard to verify that with constant probability a(H) will be at least ps. This 
follows from the fact that the expected size of H N T is at least ps. However, if 
a(G) < pn, and this is our only assumption on G, we cannot hope for a positive 
bound on the probability that a(H) < ps (unless s is equal to n). 

Thus, in the latter case, we strengthen our assumption on G to hypergraphs 
G which not only satisfy a(G) < pn but are also far from having an independent 
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set of size pn (we defer defining the exact notion of “far” until later in this dis- 
cussion). That is, given a hypergraph G which is far from having an independent 
set of size pn, we study the minimal value of s for which (with high probability) 
a random subgraph of size s does not have an independent set of size ps. This 
question (and many other closely related ones) have been studied in (2-uniform 
hyper) graphs in [GGR98] under the title of property testing. 


Property Testing: Let C be a class of objects, and P a property of objects from 
C. Property testing addresses the problem of distinguishing between elements 
c € C which have the property P and elements that are far from having the 
property P. The aim is to construct efficient (randomized) distinguishing algo- 
rithms that sample the given element c in relatively few places. The notion of 
property testing was first presented by Rubinfeld and Sudan in [RS96] where 
the testing of algebraic properties of functions was addressed. Goldreich, Gold- 
wasser, and Ron [GGR98] later initiated the study of combinatorial objects in 
the context of property testing. In their work they studied (2-uniform) graphs 
and considered several fundamental combinatorial graph properties related to 
the independence number, chromatic number, size of maximum cut, and size of 
the maximum bisection of these graphs. Since, many papers have addressed the 
notion of property testing, both in the context of functions and in the combina- 
torial setting (e.g. see surveys [Gol98,Ron01,FisO1)). 

Property testing of hypergraphs has also been studied in the past. Czu- 
maj and Sohler [CS01] initiated this line of study when analyzing the prop- 
erty of being @ colorable. Colorability, and other properties of k-uniform hy- 
pergraphs (that can be phrased as a Max-k-CNF formula) were also studied in 
[AdIVKK02,AS03]. In this work, we consider testing the independence number of 
hypergraphs. To the best of our knowledge, this property has not been addressed 
in the past (in the context of hypergraphs). 


Testing the Independence Number: Goldreich, Goldwasser, and Ron [GGR98] 
study property testing of the independence number of (2-uniform) graphs. In 
[GGR98] a graph G of size n is said to be ¢-far from having an independent set 
of size pn if any set of size pn in G has at least en? induced edges. It was shown 
in [GGR98] that if G is -far from having an independent set of size pn then 
with high probability a random subgraph of size s = slg Gae, for a sufficiently 
large constant c, does not have an independent set of size ps. The sample size 
$s was later improved in [FLS02] to ob log (£) (again c is a sufficiently large 
constant). It is not hard to verify that this implies a (two-sided error) property 
testing algorithm for the independence number of G. Namely, given a graph G, 
one may sample a random subgraph H of G of size s, and exhaustively compute 
a(H). On one hand, if G happens to have an independent set of size pn, then 
with some constant probability pı the independence number of H will be at least 
ps (recall that the expected value of a(H) is at least ps). On the other hand, if 
G is £-far from having an independent set ofsize pn then, as mentioned above, 
with at most some small probability p2 (which is set to be smaller than pı} it is 
the case that a(H) > ps. 
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Our Results: In this work, we address the study of hypergraphs in the above 
context. Namely, given a k-uniform hypergraph G = (V,E) of size n, we show 
that if one must remove at least enë edges of G in order for the remaining 
hypergraph to have an independent set of size pn (i.e. G is £-far from having an 


independent set of size pn) then a random subgraph H of G of size s x c(k) Se 
satisfies a(H) < ps with high probability. Here e(k) depends on k alone, which 
implies that the sample size s is independent of n. We also show a lower bound 


for the size of s of value c’ oO (where c’(k) is yet an additional function 
which depends on k alone). 


Definition 1 Let A be a subset of V. Define E(A) to be the set of edges in the 
hypergraph induced by A. 


Definition 2 Let p < 1. A k-uniform hypergraph G = (V,E) is said to be 
(p,€)-connected iff every subset A of V of size pn satisfies |E(A)| > en" (i.e. the 
number of edges in the subgraph induced by A is greater than en"). 


Theorem 3. Let G be a k-uniform hypergraph. Let H be a random sample of 
2k 
G of size s > c2*k!& log (£) for a large constant c. 


1. If G has an independent set of size pn, then with probability > 1/4 the 
subgraph H will have an independent set of size ps. 

2. IfG is (p,€)-connected then with probability < 1/20 the subgraph H will have 
an independent set of size ps. 


A few remarks are in place. First of all, in the above theorem we consider only 
hypergraphs which are k-uniform. Theorem 3 can be extended to hypergraphs 
in which any edge is of size at most k. This follows from the fact that any 
sufficiently large hypergraph which is (p,€)-connected with edges of size k and 
smaller contains a subgraph (on the same vertex set) which is k-uniform and 
(p,€/2)-connected. Secondly, throughout this work we analyze the properties of 
random subsets H which are assumed to be small. Namely, we assume that the 
value of s (as stated in Theorem 3) and the parameters p, € and n satisfy (a) 
s < c/n and (b) s < cpn for a sufficiently small constant e. These assumptions 
typically hold in the context of property testing. Finally, the constants 1/4, and 
1/20 presented above are not tight (we have not made an attempt to find the 
tightest possible constants). 


Proof Techniques: Given a k-uniform hypergraph G with vertex size n which 
is (p,€)-connected, the bulk of our work addresses the study of the minimal 
value of s for which a random subgraph H of G satisfies a(H) < ps with high 
probability. This is done by analyzing the probability that a random subset H of 
G of size s satisfies a(H) < ps (as a function of s, p and £). Our proof structure 
is strongly based on that appearing in (the journal version) of [FLSO2], and 
proceeds as follows. Given a sample size s, we start by bounding the probability 
that a random subset R of G of size £ > ps is an independent set. As any subset 
R of H is random in G, one may bound the probability that a( H) > ps using the 
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standard union bound on all subsets R of H of size > ps. We analyze this naive 
strategy and show that it only suffices to bound the probability of a slightly 
stronger condition than the condition a(H) > ps. Namely, using this scheme, we 
bound the probability for which a(H) > 3ps (instead of exactly ps). The naive 
strategy is then enhanced in order to bound the probability that a(H) > dps as 
a function of 6 for any ô > 1. This now suffices to bound the probability that 
a(H) > ps (using a few additional ideas). 

Applying the proof technique of [FLS02] to the case of k-uniform hypergraphs 
involves several difficulties. Roughly speaking, these are overcome by considering 
for each vertex v in the given graph G, a weighted set system of all subsets of V 
which share an edge with v. This set system plays the role of the standard “set 
of neighbors” considered when studying (two-uniform) graphs. Specifically, for 
all vertices v we consider the set system A, consisting of sets a = {u1,..., ui} 
for which there exists an edge e in G which includes both the vertex v and the 
set a. The weight applied to each subset is fixed to be a function of the subset 
size. Ideas along this line have been used in the past in the study of hypergraphs 
(e.g. [ASO3]). 


Organization: The remainder of the paper is organized as follows. In Section 2 
we present the proof of Theorem 3. In Section 3 we present our lower bounds on 
the sample size s. As mentioned above, our proof uses many of the arguments 
presented in [FLSO2]. During our presentation, we repeat quite a few of these 
arguments, in aim to make the paper comprehensible. Due to space limitations, 
some of our results appear without detailed proof. Most of the details can be 
found in an extended version of this paper [Lan03]. 


2 Proof of Theorem 3 


In this section, we prove Theorem 3. The proof of Theorem 3 (1) is straight- 
forward and (due to space limitations) is omitted. The proof of Theorem 3 (2) 
follows. 


2.1 The Naive Scheme 


Let G = (V, E) be a (p,€)-connected k-uniform hypergraph. In this section we 
study the probability that a random subset R of V of size £ is an independent 
set. We then use this result to bound the probability that a random subset H 
of G of size s has a large independent set. 

We would like to bound (from above) the probability that R induces an 
independent set. Let {ri,...,r¢} be the vertices of R. Consider choosing the 
vertices of R one by one, such that at each step the random subset chosen so 
far is Ri = {ri,...,7;} and the vertex rj; is chosen from V \ R;. Assume 
that at some stage R; is an independent set. We would like to show (with high 
probability) that after adding the remaining vertices {ri41,...,72} to Ri, the 
final set R will not be an independent set. 
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The vertices in V \ R; that cannot be added to R; are exactly the vertices 
v that share an edge with some k — 1 vertices in R; (namely, these vertices will 
result in a set R,,; that is no longer independent). Let N(R;) = N; be the set of 
such vertices in V \ R;, and let I(R;) = I; be V \ Ni. Consider the next random 
vertex ri+ı € R. If rj, is chosenfrom N; then it cannot be added to Ri, and we 
view this round as a success regarding the set R;. Otherwise, ri+ı happens to be 
in I; and can be added to R;. But if the addition of rj41 to R; happens to add 
many vertices to N;, we also view this round as a successful round regarding R;. 

Motivated by the discussion above, we continue with the following definitions. 
For each subset R; we define the following weighted set systems of subsets of V. 
Let RES(R;,1) (for restrict) be the set of singletons {v} that share an edge with 
vertices in R;. Namely, {v} E€ RES(R;,1) iff there exists vertices {wi,...,we-1} 
in R; such that there is an edge {v,wi,...,we—-1} in E. RES(R;,1) is exactly 
the set N; defined above. Define the weight of each element in RES(R;,1) as 
n*-?, Let RES(R;,2) be the set of pairs of vertices v1, v2 which (together) 
share an edge with vertices in Rj. Namely, {v1, v2} E€ RES(R;,2) iff there exists 
vertices {w1,...,W-2} in R; such that there is an edge {v1, v2, wi,...,Wk—2} 
in E. Define the weight of each element in RES(R,,2) as n*~3. Similarly for 
each 7 € {1,...,k — 1} let RES(R;,7) be the set of subsets {v1,...,v;} of V 
of size 7 that share an edge with vertices in R;. Define the weight of each ele- 
ment in RES(R,,7j) as n*-3-}, Finally let RES; = RES(R,) be the union of 
the sets RES(R;,7) where j € {1,...,k —1}. Let | RES(R,,7)|| (\|RES;|]) de- 
note the weight of elements in RES(R;,7) (RES;). Notice that |RES(R;,7)|| < 
(j)nk-I-? <n*~? and that ||RES;|| < ee, jes kn, 


Definition 4 Let the normalized degree w.r.t. Ri of a vertex v € V be the 
amount on which v restricts upon R;: 


d,(Ri) = ||RES(R; U {v})|| — | RES(R)|| = || RES(R; U {v}) \ RES(R)I. 


In the above notice that RES(R;) C RES(R; U {v}). We call a vertex v in V 
1 


heavy with respect to R; (or R;-heavy for short) if dẹ(R;) > ray 

Each subset R; of V now defines the following partition (LL, H, N;) of V 
and the set RES;. Let RES; and N; be as defined as above. Let [; = V \ Nj. 
I; is now partitioned into two parts: vertices in J; with low normalized degree 
(w.r.t. Ri), denoted as the set DJ;, and vertices with high normalized degree, 
denoted as HI;. Namely LI; is defined to be the pn vertices of J; with minimal 
normalized degree and HT; is defined to be the remaining vertices of J;. Ties are 
broken arbitrarily or in favor of vertices in R; (namely, vertices in R; are placed 
in LI; before other vertices of identical degree). If it is the case that |J;| < pn 
then LI; is defined to be J;, and HJ; is defined to be empty. 

We define the partition corresponding to Rg = ¢ as (Llp, H Io, No), where 
LIo are the pn vertices of G of minimal normalized degree, H To are the remaining 
vertices of G, and No isempty. RE So is also defined to be empty. 

Notice, using this notation, that the subset R; is an independent setiff R; C 
I,. Moreover, in this case R; C LI; (all vertices of R; have normalized degree 0). 
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Furthermore, each vertex r; in an independent set R = Rg = {rj,...,r¢} satisfies 
ri € J;-1. 

We are now ready to bound the probability that a random subset R = 
{ri,...,Te} of G is independent. Let R; = {ri,...,ri}, and let (LI;, HL, N;) 
be the corresponding partition of V defined by R;. Consider the case in which 
R is an independent set. As mentioned above, this happens iff for every 7 the 
vertex r; is chosen to be in -1 = L-1 U HI,;_,. We would like to show that 
this happens with small probability (if £ is large enough). 

Consider the set RES; as we proceed in the choice of vertices in R. Initially, 
the subset RESọo is of weight 0, and it gets larger and larger as we proceed in 
the choice of vertices in R. Each vertex in r; € Hl;—ı increases the weight of 
RES;—ı substantially, while each vertex in LI;—-ı may only slightly change the 
weight of RE Si—ı. In the following, we show that there cannot be many vertices 
ri E€ R that happen to fall into HI;—1. We thus turn to consider vertices r; that 
fallin LI;—ı (there are almost @ such vertices). The size of LJ; is bounded by 
pn. Hence, the probability that r; € LI, is bounded by p (by our definitions 
Ri-ı C Lli—ı and the vertex r; is random in V \ Ri—-1). This implies that the 
probability that R is an independent set is roughly bounded by p£. Details follow. 


Lemma 1. Let G be a (p,£}-connected hypergraph. Let R = {r1,..., re} bea set 
in G. The number of vertices r; which satisfy r; € HI;—ı is voned byt = Fhe 


Proof. We start with the following claim which is the main technical contribution 
of this work. 


Claim 5 Let R; be as defined above, and let (LI;, H1I;,N;) and RES; be its cor- 
responding partition and set system. Let l; = LLU HI. If G is (p,€)-connected 
then, every vertex in HI; is R;-heavy. 


Proof. Assume that LI; = pn (otherwise HJ; is empty and the claim holds). By 
our assumptions on G we have that LJ; induces at least en* edges. Let E(LI;) be 
the set of these edges, and let m be the size of E(LJ;). To simplify our notation, 
let LI = LR, HI = HL, N= N; and R= R,. a that the index 7 is free, we 
use it in the following new context. For 2 € oe — 1} consider the following 
sequence of weighted i-uniform hypergraphs H cee E;) which all have vertex 
set LI. The edge set E; is defined to be all ices of size 2 of edges in E(LI) 
(here and throughout our work we consider a (hyper) edge of size k as a subset 
of vertices of size k). For example, each edge in E(LJ) induces (5) edges in Ha, 
& ) edges in H3, and so on. The weight of an edge e; in H; is equal to the number 
of edges e in E(LI) which satisfy e; C e. 

Recall the sets RES(R,7) and RES(R). Our goal is to prove the existence 
of a vertex v € LI which is R-heavy (this will imply our assertion). Namely a 
vertex v which when added to R will significantly increase the weight of RES(R). 
Formally we are looking for a vertex v € LI which satisfies || RES(R U {v})]] — 
|RES(R)|| > SRI s En*k-1, Notice that the weight of a subset of vertices with 


respect to RES(R) may (and usually does) differ from its weight with respect 
to H;. To avoid confusion we denote the weight of a subset e with respect to 
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RES(R) as wr(e). Recall that every (partial) edge e of size i in RES(R) has 
weight wr(e) = n*-*-!, The weight of an edge e in H; will be marked as wy, (e). 

For each ¿ we now consider the weight (with respect to H;) of the edges in 
H; which are in RES(R). Denote this weight as res;. Notice that res; = 0 (by 
our construction LI N N = ¢, which implies that singletons {v} C LJ are not in 
RES(R)). We consider the following cases. 


Case 1: We start by assuming that res,_, < (k — 1)m. As the total weight of 
edges in Hk—ı is km, we conclude that the weight of edges in Hy,_; that are not 
in RES(R) is at least m. Let v be a vertex in LI. We say that an edge e in 
Hy, is a v-edge if there exists an edge in E(LJ) consisting of the union ofe and 
{v}. Notice that an edge e in Hy_; of weight wy,_,(e) is actually a v edge for 
wyH,_,(e) distinct vertices v. As each edge in Hy_; is a v-edge for some vertex 
v in LI, we have the existence of a vertex v in LI with at least oa (distinct) 
v-edges that are not in RES(R). The weight of each such v-edge in RES(R) is 
1. Furthermore, each such v-edge appears in RES(RU {v}) implying that 


|RES(RU {v})]| — | RES(R)|| > a > snk 


which in turn implies the v is R-heavy. 


Cases 2 to k — 1: Starting at i = k — 1 and iteratively continuing until 2 = 2 
consider the following cases. From the previous step, we may assume that res; > 


fe ee. We now also assume that res;_1 < see et Consider an edge 
e = (v1,...,v;) in H; of weight wy, (e) which contributes to res;. Each such edge 
inducesi edges in H;—1: {e},...,e,} (each edge obtained by removing one vertex 


frome). We are interested in bounding (by below) the weight of edges e as above 
with corresponding edges e} which are not in REBR) (here j € {1,...,7}). 

By our construction, the weight of any edge e’ in H;i—ı equals the oo of 
edges in G that include e’. It is not hard to verify that this equals Root times 
the weight of edges in H; which include e’. We conclude that the weight of edges 
ein H; which are in RES(R) with some corresponding edge e’ in H;~1 which is 
also in RES(R) is bounded by res;_1(k —i+1). This leaves us with 








k-1 
res; —(k—i+1)res;_; > MS i = = 
edges e which contribute to res; for which e; for j € {1,...,i} are not in 


RES(R). 

We now conclude the existence of a vertex v € LI which is adjacent to (i.e. 
is included in) at least the weight of A edges in H; which appear in 
RES(R) such that their corresponding edges in H;_; are not in RES(R). The 
weight of each edge in H; is bounded by ie TEZ (on)*-*. Thus there are at least 
a (Eh distinct edges in H; adjacent to v with corresponding edges 
in H;i—ı that are not in RES(R). 
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We will now show that v is R-heavy. Consider one of the distinct edges 


e = (v,U1,...,Us~1) aS discussed above. By our assumption the set {v4,..., vi-1} 
is not in RES(R). The edge e appears in RES(R) implying that there exists 
vertices {wy ,...,we—i} (in R) such that (v, v1,...,Ui—1, W1,---, We-i) IS an edge 


in the original hypergraph G. This in turn implies that {v,,...,v;-1} will be 
included in RES(R U {v}). As each set of size i — 1 in RES(R) has weight 
wp = n*-*, and there are at least 5 


adjacent to v, we conclude that 


BEAT oa) = T distinct edges of interest 


1 (k=1) 1) € nko} 
2k—i (k — i)! p l 





|RES(RU {v})|| - | RES(R)|| > 


Now to prove our lemma, consider the subsets R; = {ri,...,7;} and their 
corresponding partitions (LL, HI;,N;). Let J; = LI; U HI. We would like to 
bound the number of vertices r; that are in HJ;_1. Conon a vertex r; in HJ;. 
By Claim 5, its normalized degree w.r.t. R; is at least AION Enk-1. RES(¢) 


is initially empty, and for any i, RES(R;) is of weight at most a : (;)n ERREN 
kn*~". Each vertex r; € HI;—1 increases | RES(R;-1)|| by at least zeggzay $n"? 


k e e e e 
We conclude that there are at most ae vertices r; in R which are in HJ;_1. 


Let ô be a large constant. “Plugging?” Lemma 1 into the proof appearing in 
(the journal version of) [FLS02], we are now able to bound the probability that 
a random subset H of G of size s has an independent set of size > dps. This is 
done in two steps, first we bound the probability that a random subset R of G 
is independent. Then, using the union bound, we obtain the result mentioned 
for H. In the upcoming Section 2.2 we present the refined proof technique of 
[FLS02] (which gets rid of the parameter 6) applied to the case of hypergraphs. 
Namely, we will bound the probability that a random subset H of G of size s 
has an independent set of size > ps. 


Lemma 2 ({FLS02]). Let G be a (p,€)-connected hypergraph. Let t be as in 
Lemma I. Let £ > 2t. The probability that £ random vertices of G induce an 


independent set is at most 
TEAN 
PNG 0 


Corollary 6 ({FLS02]) Let G be a (p,¢€)-connected hypergraph. Let t be as in 
Lemma 1. Let H be a random sample of G of size s. Let 6 > e, and let c bea 
sufficiently large constant. If s > ct aC Le) then the probability that a(H) > ps 


is at most ($ eae 


2.2 An Enhanced Analysis 


Recall the proof technique from Section 2.1. We started by analyzing the prob- 
ability that a subset R of H of size £ is an independent set. Afterwards we 
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bounded the probability that a(H) > dpn (using the standard union bound on 
all subsets R of H of size greater than £ = dpn). In this section we enhance 
the first part of this scheme by analyzing the probability that a subset R of H 
of size is a maximum independent set in H (rather than just an independent 
set of H). Then, as before, using the standard union bound on all large subsets 
R of H, we bound the probability that a(H) > ps. We show that taking the 
maximality property of R into account will suffice to prove Theorem 3 (2). 

Let H = {h1,..., hs} be s random vertices in G. We would like to analyze 
the probability that a given subset R of H of size £ is a maximum independent 
set. Recall (Section 2.1), that the probability that R is an independent set is 
bounded by approximately pf. An independent set R is a maximum independent 
set in H only if adding any other vertex in H to R will yield a set which is no 
longer independent. Let R = Re be an independent set, and let (LIe, HIe, Ne) 
be the partition (as defined in Section 2.1) corresponding to R. Consider an 
additional random vertex h from H. The probability that RU h is no longer 
an independent set is approximately |Ne|/n (here we assume that |R| is small 
compared to n). The probability that for every h € H \ R the subset RU {h} 
is no longer independent is thus ~ (Nel / n)". Hence, the probability that a 
given subset R of H of size @ is a maximum independent set is bounded by 
approximately p* (| N| In. This value is substantially smaller than p*iff |Ne| 
is substantially smaller than n. We conclude that it is in our favor to somehow 
ensure that |N¢| is not too large. We do this in an artificial manner. 

Let R = {rj,...,re} be an independent set, let R; = {r1,..., ri}, and let 
(LI; HI;,N;) be the partition (as defined in Section 2.1) corresponding to Ri. 
Let RES; be the set system corresponding to R. Roughly speaking, in Sec- 
tion 2.1, every time a vertex r; was chosen, the set system RES; and N; was 
updated. If r; was chosen in HJ;_;, then RES;_, increased substantially and 
N;—ı potentially also grew substantially, and if r; was chosen in LJ;-1, both 
RES;—, and N;—1 only slightly changed. We would like to change the definition 
of the partition (LI, HI;,N;,) and of RES; corresponding to R; as to ensure 
that N; is always substantially smaller than n. This cannot be done unless we 
relax the definition of N;. Recall that N; was defined (in Section 2.1) to be the 
set of vertices v which share an edge with vertices in R;.Specifically, N; is the 
set of vertices v for which there exists vertices {w1,...,Wk—-1} in R; such that 
there is an edge {v,wy,...,Wx-1} in E. In this section N; will only include a 
subset of these vertices (a subset which is substantially smaller than n). Namely, 
in our new definitions RES;_; and Nj; will be changed only if r; was chosen 
in HI;—1. In the case in which r; € L-1 U N;_1, we do not change N;—1 at all. 
As we will see, such a definition will imply that |N;| < (1 — p)s, which will now 
suffice for our proof. 


A New Partition and Set System: Let H = {hy,...,hs} be a subset of V. Let 
R; = {r1,... ri} be a subset of H of size i. In the previous section, the subset 
R; defined a partition (LI, HI;, Ni) of V and a set system RES;. In this section, 
foreach ¿ we will define a subset Ê; of R;, and a new partition and set system. 
The new partition and set system corresponding to R; will be defined similarly 
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to those defined in the previous section with the exception that R, will play the 
role that R; played previously. The set R;, the new partition (LJ;, HI;, N;), and 
the set system RES; are defined as follows (as before, let J; = LI; U HI). 


1. Initially Ro = Ro = og, Llo is the pn vertices in V of minimal normalized 
degree w.r.t. R;, HI = V \ LIp, and No = ¢. In the above, ties are broken 
by an assumed ordering on the vertices in V. RESp = RES(Ro) is defined 
to be empty. 

2. Let Ê;, (LI, HI;,N;) and RES; be the sets corresponding to Ri, let ri+4 
be a new random vertex. Let Ri+ı = Ri U {ri+1}, we now define the sets 
Rist, (Lli41, Hliti1, Niz1) and RES;41. Let N(ri+1) be the set of vertices 
which share an edge with vertices in R; U 7,41. We consider the following 
cases: (a) If riz, € LI; or riz, € N; then the sets corresponding to Ri+ı 
will be exactly those corresponding to R;. Namely, Riz; = Ri, LIi41 = Lh, 
Alia, = HL, Nig, = Ni, and RES;,; will be defined as RES;. (b) If 
ri41 € HI; then we consider two sub-cases: (b1) If |N;UN(ri4i)]| < (1—p)n, 
then Ri+ı = R; U {ri+1}, RESj44 = RES(Ri+1), and L124; Ali+1, N41, 
are defined as in Section 2.1. Namely, Niz1 = Ni U N(ri+1). Lig1 is defined 
to be V \ Nii. Li, is defined to be the pn vertices of f;41 with minimal 
normalized degree w.r.t. Ê;+1. Finally, HJ;41 is defined to be the remaining 
vertices of J;41. Ties are broken by the assumed ordering on V. (b2) If 
IN; U N(rig1)| > (1 — p)n, then let N(ris1) be the first (according to the 
assumed ordering on V) (1 — p)n — |N;| vertices in N (Ti41) and set Ni+ı = 
N; U N(ri41). Furthermore, set LJ;41 to be the remaining pn vertices of 
G, and HI; to be empty. Notice that in this case |Ni+;| is of size exactly 
(1 = p)n. Finally, let Riz. = R; and RESj41 = RES;. 


A few remarks are in place. First of all it is not hard to verify that the 
definition above implies |N;| < (1—p)n for allz € {1,..., £}. Secondly, due to the 
iterative definition of our new partition, the sets corresponding to the subsets 
R,; depend strongly on the specific ordering of the vertices in Ri. Namely, in 
contrast to the partitions (set systems) used in Section 2.1, a single subset R 
with two different orderings may yield two different partitions (set systems). For 
this reason, in the remainder of this section, we will assume that the vertices of 
H are chosen one by one. This will imply an ordering on H and on any subset 
R of H. The partitions we will study will correspond to these orderings only. 

Finally, in Section 2.1, an (ordered) subset R = {r1,..., re} was independent 
iff Vi r; € I;_1 (according to the definition of [;-1 appearing in Section 2.1). In 
this section, if R is independent then it still holds that Vz r; € J;-1. However, 
it may be the case that Vi r; € 4:1 but R is not an independent set. In the 
remainder of this section, we call ordered subsets R for which Yi r; € [;-1 free 
sets. We analyze the probability that a random ordered subset H of V of size s 
does not have any free sets R of size larger then ps. This implies, that H does 
not include any independent sets of size ps. 

We now turn to prove Theorem 3 (2). Roughly speaking, we start by analyz- 
ing the probability that a random subset R is a free set. We then analyze the 
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probability that a given subset R in H is a maximum free set. Finally, we use 
the union bound on all subsets R of H of size > ps to obtain our results. 

We start by stating the following lemma which is analogous to Lemma 1 from 
Section 2.1. The main difference between the lemma below (and its proof), and 
that of the previous section is in the definition of the partition (LI; HL, N;), 
and the set system RES;. Proof of the lemma is omitted. 


Lemma 3. Let G be a (p,€)-connected hypergraph. Let R = {r1,...,re} bean 
ordered set in G of size £. The number of vertices r; which satisfy r; E€ HI; is 


bounded by t = ae 


We now address the probability that a random subset R of H is a maximum 
free set. We then use the union bound on all subsets R of H of size > ps (and 
a few additional ideas) to obtain our main result. Due to space limitations, the 
proof of the following Lemma and Theorem is omitted. The interested reader 
can find their proofs in [Lan03] (or proofs of similar nature in [FLS02]). 


Lemma 4. Let G be a (p,€)-connected hypergraph. Let t be as in Lemma 3. Let 
£ > 2t. Let H be an ordered random sample of G of size s > £. The probability 
that a given subset R of H of size £ is a maximum free set is at most 


el i 
g s—£ 
pit E a EE, 
(=) ( ) 


Theorem 3 (2). Let G be a (p,€)-connected hypergraph. Lett be as in Lemma 3. 
Let H be a random sample of G of size s. Let c be a sufficiently large constant. 
If s > ct log (£) then the probability that H has an independent setof size 
> ps is at most e~ 2, 


3 Lower Bounds on the Sample Size s 


Roughly speaking, Theorem 3 (2) states that given a (p, €)-connected hypergraph 
k 

G, a random sample H of G of size s proportional to 2k kl ee (or larger) will not 

have an independent set of size ps (with high probability). We now continue to 

study the minimal value of s forwhich a(H) < ps with high probability, and 


2k—1 , 
present a lower bound of EO on the size of s. 


Lemma 5. Letn be a sufficiently large constant. Let p > 0 and ¢ > 0 satisfy 
(la) € << a (b) p*®-*/e* << n and (c) k? << pn (herea << b if for a 
sufficiently small constant ô > 0 it holds thata < 5b). There exists a graph G on 
n vertices for which G is (p,€)-connected, and with probability > 1/20 a random 


subgraph H of size s = Iapa will have an independent set of size ps. 


Proof. Consider the k-uniform hypergraph G = (V, E) in which (a) V| = n, 


(b) V consists of two disjoint sets A and V\A, where A is of size (1 — 2E) pn, 


and (c) the edge set E of G consists of all subsets of V of size k except those 
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included in A (namely, A is an independent set). On one hand, every subset 
of size pn in G induces a subgraph with at least enë edges (implying that G 
is (p,€)-connected). On the other, let H be a random subset of V obtained by 
picking each vertex independently with probability Lfe. The expected size 


of H is s = fae. In the following, we assume H is exactly of size s, minor 
modifications in the proof are needed if this assumption is not made. The set 
HNA is an independent set in the subgraph induced by H. The expected size of 
HnAis (1- RE) ps. Let N(0,1) denote a standard normal variable. It can be 
seen using the central limit theorem (for example [Fel66]) that for our choice of 
parameters, the probability that |H N A| is greater than (1 — 2ks) ps+./ps is 
at least Pr [N(0,1) > 3/2] > 4. In such a case the size of HN A will be greater 


than (1 — an )ps + ,/ps = ps for our value of s. Hence implying the lemma. 
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Abstract. We describe a deterministic algorithm that, for constant k, 
given a k-DNF or k-CNF formula ¢ and a parameter e, runs in time linear 
in the size of y and polynomial in 1/¢ (but doubly exponential in k) and 
returns an estimate of the fraction of satisfying assignments for y up to 
an additive error €. This improves over previous polynomial (but super- 
linear) time algorithms. The algorithm uses a simple recursive procedure 
and it is not based on derandomization techniques. It is similar to an 
algorithm by Hirsch for the related problem ofsolving k-SAT under the 
promise that an €-fraction of the assignments are satisfying. Our analysis 
is different from (and somewhat simpler than) Hirsch’s. 

We also note that the argument that we use in the analysis of the algo- 
rithm gives a proof of a result of Luby and Velickovic that every k-CNF 


is “fooled” by every 6-biased distribution, with 6 = 1/202"). 


1 Introduction 


We consider the following problem: given a k-CNF formula p and a parameter 
€, approximate within an additive error € the fraction of satisfying assignments 
fory'. 

The problem is easy to solve using randomization: just generate O(1/e7) 
assignments at random and then output the fraction of assignments in the sample 
that satisfies y, and the question is whether efficient deterministic algorithms 
exist. 

We also consider the related problem of finding a satisfying assignment for 
yy under the promise that an € fraction of assignments are satisfying. Again, 
we are interested in deterministic algorithms, and the problem is easy to solve 
probabilistically, since after picking O(1/e) assignments at random it is likely 
that one of them satisfies the formula. 

One can consider the approximate counting problem as the problem of de- 
randomizing two-sided error algorithms implemented by depth-two circuits. The 


* Supported by NSF grant CCR-9984703, a Sloan Research Fellowship and an Okawa 
Foundation Grant. 

l Note that an algorithm achieving additive approximation € for k-CNF immediately 
implies an algorithm achieving the same additive approximation for k-DNF. Also, 
achieving multiplicative approximation (1 + £) for k-DNF reduces to achieving ad- 
ditive approximation ¢2~*, since a satisfiable k-DNF is satisfied by at least a 1 J gs 
fraction of assignments. 
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problem of finding a satisfying assignment for p under the promise that that 
there is a large number of such assignments can be seen as the problem of de- 
randomizing one-sided error algorithms implemented by depth-two circuits. 
These problems were first studied by Ajtai and Wigderson [AW89]. Using 
derandomization techniques (specifically, t-wise independence) they give an al- 


gorithm for the counting problem running in time O(n* + 20080/ e))?") and an 


algorithm for the satisfiability problem running in time O(n*2" e(/2)), They 
also give sub-exponential time algorithm for the counting problem for functions 
computed by AC® circuits’. 

The algorithm of Ajtai and Wigderson for k-CNF could be improved by using 
almost t-wise independent distributions, for example the small bias distributions 
of [NN93], instead of distributions that are perfectly t-wise independent. For 
constant €, this would improve the running time to roughly n - (logn)?™ . 


k2 
2k" for both the approximate counting problem and the satisfiability problem. 


Almost t-wise independent distributions were introduced after the publication 
of [AW89]. 

Nisan [Nis91] and Nisan and Wigderson [NW94] construct a pseudorandom 
generator that fools constant-depth circuits and that has poly-logarithmic seed 
length. As a consequence, they achieve n“% nyo time algorithms for the count- 
ing and satisfiability problems for AC® circuits. 

Luby, Velickovic and Wigderson [LVW93] optimize the constructions of Nisan 
and Wigderson [Nis91,NW94] to the case of depth-2 circuits, thus solving the 
counting and satisfiability problem in time n°(@°¢™”) for general CNF and DNF. 
Luby and Velickovic [LV96] show how to reduce an arbitrary CNF and DNF to 
a formula in a simplified format, and show that the counting and satisfiability 
problems can be solved in polynomial time for k-CNF even if k = O((log n)1/8) is 
more than a constant. Luby and Velickovic [LV96] also also present an improved 
derandomization of general CNF and DNF that runs in slightly super-polynomial 


time nO(2V 151"), Finally, Luby and Velickovic also show that every k-CNF is 
“fooled” by every distribution that is close to being t(k)-wise independent, where 
t(k) = 2°(F2"), (A distribution X “fools” a function f() if the probability that 
f(X) =1 is approximately the same as the probability that f() equals 1 under 
the uniform distribution.) 

Hirsch [Hir98] shows how to solve the satisfiability problem for k-CNF in time 
O(Lk(2/e)?“*)), where L < nk is the size of the formula and B(k) is a function 
for which a closed formula is not given, but that seems to grow exponentially in 
k. Hirsch’s algorithm does not use derandomization techniques. 

In this paper, we show how to solve the approximate counting problem and 
the satisfiability problem in time O(L(1/e)(24)*?" ). 

Our algorithm is based on the following simple observation: given a k-CNF 
y, then for every fixed c, either we can efficiently find a set of < kc variables 


2 An AC? circuit is a circuit of constant depth and polynomial size with unbounded 
fan-in AND and OR gates. A CNF formula is a depth-two AC® circuit, and so is a 
DNF formula. 
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that hits all the clauses, or we can efficiently find > ¢ clauses over disjoint sets 
of variables. In the former case, we can try all assignments to those variables, 
and recurse on each assignment, thus reducing our problem to 2** problems on 
(k — 1)-CNF instances; in the latter case, less than a (1 — 1/2*)* fraction of 
assignments can satisfy y, and thus O is an approximation within an additive 
error (1 — 1/2*)* of the fraction of satisfying assignments for y. Fixing c to be 
2* In 1/e gives us the main result. 

Using the same recursive approach adopted in our algorithm, we give a 
slightly different proof of the result of Luby and Velickovic about k-CNF being 
fooled by almost t-wise independen distributions. We show that every k-CNF 
is well approximated, in a certain technical sense, by a decision tree of depth 
t = O(k2*), and it is well known that functions that are well approximated (in 
the above technical sense) by a decision tree of depth £ cannot distinguish the 
uniform distribution from a distribution that is approximately t-wise indepen- 
dent. 


2 The Recursive Algorithm 


We describe the algorithm only for the case of k-CNF. As discussed in the 
introduction, an algorithm for k-DNF is an immediate corollary of the algorithm 
for k-CNF. 

We use the following simple fact. 


Lemma 1. There is an algorithm that, on input a k-C'NF formula y and a 
parameter t, runs in time linear in the size of œ and then it returns either a set 
of t clauses over disjoint sets of variables, or a set S of at most k(t —1) variables 
such that every clause in the formula contains at least one variable from S. 


Proof. (Sketch) Consider the k-uniform hypergraph H that has a vertex for 
every variable and an hyperedge for every clause. It is easy to find a maximal 
matching in H in linear time, that is, a set S of clauses over disjoint sets of 
variables and such that every other clause in y shares some variables with some 
clause in S. If |S| > t, then we return t of the clauses in S. Otherwise, we return 
the set of < k-|S| < &k(t-—1) variables that occur in the clauses of S. Such a set 
of variables clearly “hits” all the clauses of ¢. o 


The algorithm works as follows: given y and €, 


— If y is a 1-CNF, that is, it is just an AND of literals, then we output O if 
there are two inconsistent literals and 2~° where c is the number of distinct 
literals, otherwise. This procedure is exact and can be implemented in linear 
time. 

— Otherwise, we let t be the smallest integer such that (1 — 1/2*)**! < e, 
so that t < 2*(In1/e), and we run the algorithm of Lemma 1 on ẹ with 
parameter t + 1. 
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e Ifthe algorithm of Lemma 1 finds +1 clauses C),...,C¢41 over disjoint 
sets of variables, then it is clear that the probability that ¢ is satisfied 
by random assignment is at most £, and we return the value O as our 
approximation. 

e If the algorithm of Lemma 1 finds a set V of at most tk < k2* In(1/e) 
variables that hit all the clauses, then, for every assignment a to the 
variables V, define Ya to be the formula obtained from ¢ by substituting 
the assigment into the variables. Note that pa is a (k —1)-CNF formula. 
We recurse on each of the pa with parameter €, and take the average 
of the results. Assuming that each recursive call returns an € additive 
approximation, the algorithm returns an € additive approximation. 


If we denote by T(L,k) the running time of the algorithm for a k-CNF 
instance of size L, then we have 


T(L,1) = O(L) 


and i 
T(L, k) < O(L) + 200/2 T(L,k— 1) 


which solves to T(L, k) = O(L - 2240 1/6)2*) — O(L(1/e)02 942"), 

For the promise problem of finding a satisfying assignment under the promise 
that an £ fraction of assignments are satisfiable, we essentially use the same re- 
cursive algorithm. When we are down to 1-CNF, we find a satisfying assignment 
or fail if the instance is unsatisfiable. (Indeed, we can stop at 2-CNF.) In the 
recursive step, we fail if t is such that (1 — 1/2*)* < €. The analysis of the run- 
ning time is the same, and it is clear that at least one of the recursive branches 
produces a satisfying assignment. 

Hirsch’s algorithm is similar to the above sketch of the algorithm for the 
satisfiability promise problem, except that a different greedy strategy is used to 
pick the variables in V. The analysis is slightly different and somewhat more 
difficult. 


3  Pseudorandomness Against k-CNF Formulae 


3.1 Some Technical Preliminaries 


We begin this section with a few technical definitions. 

We denote by U,, the uniform distribution over {0,1}". If f : {0,1}” — {0,1} 
is a function and X is a distribution over {0,1}", then we say that X ¢-fools f 
if 

|Pr[f(Un) = 1] - Prif(X) = ll < € 

If F is a collection of functions, then we say that a distribution X é-fools F if 
X é-fools every function f € F. 

Our goal will be to find a distribution X that ¢-fools the class of k-CNF 
formulae over n variables, and that is uniform over an efficiently constructable 
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support of polynomial size. Then, given a k-CNF formula f, we can approximate 
Pr[f (Un) = 1] by computing the close value Pr[f(X) = 1], and we compute the 
latter by enumerating all the polynomially many elements in the support of X, 
and applying f() to each of them. We will show that e-biased distributions, 
defined below, can be used towards such goal. 

We say that a distribution X over {0,1}" is -biased [NN93] if for every 
subset S C {1,...,n} we have 


1 
9 ESPr 





D=: <5+e 


iES 


Equivalently, we can say that a distribution is -biased if it ¢-fools every linear 
function. (Where, of course, we mean linear over the field GF(2).) 


Theorem 1 ([NN93,AGHP92]). For every €, and n, there is an €-biased 
distribution over {0,1}" that is uniform over a support of size polynomial in n 
and 1/e. Furthermore, the support can be constructed in time polynomial in n 
and 1/e. 


We say that a distribution X = (X1 -+ Xn) over {0,1}", where each X; is 
unbiased, is k-wise independent if every k of the random variables X1,...,Xn 
are mutually independent. Equivalently, a distribution is k-wise independent if 
it O-fools the class of functions that depend only on k or fewer input variables. 

We say that X is é-close to k-wise independent if for every function 
g : {0,1}" — {0,1} that depends on k or fewer inputs we have 


|Pr[g(Un) = 1] — Prig(X) = ll < € 


that is, if X ¢-fools the class of functions that depend on at most k inputs. 
We say that X is -close to k-wise independent in £,, norm if for every t < k, 
forevery t indices 71,...,7 in {1,..., n} and for every t values a1,...,a2 € {0,1} 


we have 


1 1 
pe E S PriXs, Sar A AX, =a] S re 


that is, if X €-fools the class of functions that can be expressed as checking that 
a subset of at most k bits of the input equals a particular sequence of values. 

The following connection between the notions that we have described is well 
known. 


Lemma 2. Let X be an €-biased distribution over {0,1}". Then, for every k, 
X is ¢-2*/?-close to k-wise independent, and also 2¢-close to k-wise independent 
in fs norm. 


3.2 e-Biased Distribution and k-CNF Fomulae 


In this section iwe give a proof of the following theorem. 
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Theorem 2 ({LV96]). There are functions t(k,é) = O(k - 2* - log(1/e)) and 
ô(k, £) = 1/202" -log(1/e)) such that the following happens. 

Let f : {0,1}" — {0,1} be a function defined by ak-CNF formula and let X 
be a distribution over {0,1}” that is 5(k,¢)-close to t(k,€)-wise independent in 
Log norm. Then X «€-fools f. 


The application of Theorem 2 to é-biased distributions is immediate. 


Corollary 1. There is a function 5'(k,e) = 1/20?" 80/2) such that if X 
is a 6'(k,€)-biased distributions, then X €-fools every function computed by a 
k-CNF formula. 


The rest of this section is devoted to the proof of Theorem 2. 

Let f : {0,1}" — {0,1} be a function defined by a k-CNF formula p over 
variables z1,...,£n, and consider a decision tree over the variables z1,..., £n. 
Every leaf of the decision tree (indeed, every node of the decision tree) defines 
a restriction, that is, an assignment to a subset of the variables z1,...,£n. Ifa 
leaf v is at distance t from the root, then it defines an assignment to ¢ variables; 
if we pick a random assignment and then apply the decision tree to it, there is 
a probability 1/2* that we reach the leaf v. In general, for a vertex v at distance 
t from the root we define the probability of v to be 1/2*, and for a set of vertices 
such that none of them is an ancestor of any other we define the probability of 
the set as the sum of the probabilities of the individual vertices. 


Lemma 3. Let f : {0,1}" — {0,1} be the function defined by a k-CNF formula 
p and £ > 0. Let t be an integer such that (1 ~ 1/2*)* < £. Then there is 
a decision tree of depth at most tk such that: either (i) all the leaves define 
restrictions relative to which f is a constant, or (ii) all the leaves, except possibly 


a set of probability at moste, define restrictions relative to which p becomes a 
(k —1)-CNPF. 


Proof. We apply Lemma | to y with parameter t. Then we either find t clauses 
over disjoint variables or k(t — 1) variables that hit all the clauses. 

In the former case, consider the decision tree that reads all the < kt variables 
that occur in the ¢ clauses. All but an € fraction of the leafs of the decision 
tree correspond to restrictions relative to which ¢ is zero, and, in particular, is 
constant. 

In the latter case, consider the decision tree that reads all the < kt variables 
returned by the algorithm. All the leafes of the decision tree correspond to re- 
strictions relative to which ¢ is a (k — 1)-CNF. o 


We now compose the construction. 


Lemma 4. There is a function t(k,£€) = O(k2® ln(1/£)) such that for every 
f : {0,1}" — {0,1} defined by a k-CNF formula and for every € > 0 there is 
a decision tree of depth at most t(k,£) such that all the leaves, except possibly 
a subset of probability €, correspond to a restriction relative to which f is a 
constant. 
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Proof. We prove the theorem by induction. Lemma 3 proves the theorem for 
k = 1 and ¢(1,¢) = log. 1/e. For general k, start by constructing the decision 
tree for f as in Lemma 3. If all but an € fraction of the leaves of the tree make f 
become a constant, then we are done. Otherwise, every leaf of the tree defines a 
restriction relative to which f is a (k —1)-CNF, and we can apply the induction 
hypothesis to get decision trees for each of these (k — 1)-CNF. 

This argument proves the theorem for every function ¢() that satisfies 
t(1,£) = logy 1/e and t(k,e) > k2* In(1/e)+t(k—1, ©). In particular, the theorem 
is true for t(k,£) = 2k2* In(1/e). S 


To prove Theorem 2 we now only need the following simple last step, which 
is well known. 


Lemma 5. Let f : {0,1} — {0,1} be a function and T be a decision tree of 
depth t such that all but an £ fraction of the leaves of T define a restriction 
relative to which f is a constant. Let X be a distribution that is 6-close to t-wise 
independent in €4, norm. Then 


|Pr[f(Un) = 1] - Pr[f(X) = 1| <e +8- 2 


Proof. We may assume withouth loss of generality that T has 2* leaves, all at 
distance t from the root. (Otherwise, from leaves that are closer to the root, 
we read additional variables until we reach distance t. This does not change the 
properties of T assumed in the Lemma.) Let S be the set of leaves of T that 
define a restriction relative to which f is the constant 1. Then we have 


Prif(Un)=11< Elhe 


If we sample an assignment according to X, we see that for each leaf of T there 
is a probability at least 1/2° — 6 that the assignment is consistent with the leaf. 
In each of these event, f evaluates to one and, moreover, all these events are 


disjoint. We deduce 
Z 
Pr) =1)> Él- 3.12] 


and 
Pr[f(U,) = 1] —Pr[f(X) = 1) <e+6.-2° 


Similarly, we can prove 


Pr[f(X) = 1] — Pr[f(U,) =1) <e4+6-2' 


4 Perspective 


The current body of work on derandomization (see [Kab02] for a survey) strongly 
suggests that every problem (including search problems and promise problems) 
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that is solvable probabilistically in polynomial time can also be solved deter- 
ministically in polynomial time. It is then a natural research program to look 
for deterministic polynomial time algorithms for all the interesting problems for 
which only probabilistic polynomial time algorithms are known. 

After the discovery of a deterministic polynomial time algorithm for test- 
ing primality [AKS02], the most interesting algorithms to derandomize are now 
the identity test for low-degree polynomials and the approximate counting al- 
gorithms based on the Markov-Chain Monte-Carlo approach. Kabanets and Im- 
pagliazzo [KIO3] show that derandomizing the polynomial identity test algorithm 
for general arithmetic circuits implies the proof of circuit lower bounds that may 
be beyond our current proof techniques. It is not clear whether there are similar 
inherent difficulties in derandomizing approximate counting algorithms such as, 
say, the Permanent approximation algorithm of [JSVO1]. 

The problem of approximately counting the number of satisfying assignments 
for a given circuit up to a small additive error is clearly precisely the same 
problem as derandomizing every promise-BPP problem. (In particular, such an 
approximate counting algorithm would imply that NE XP Z P/poly.) It seems 
possible, however, to derandomize at least bounded-depth circuits, and, at the 
very least, depth-two circuits in polynomial time using current techniques. In 
this paper we note that a special case of this problem can be solved by a simple 
divide-and-conquer algorithm, without using derandomization techniques. 
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